aboutsummaryrefslogtreecommitdiffstats
path: root/org.eclipse.jgit/src
Commit message (Collapse)AuthorAgeFilesLines
* DfsObjDatabase.PackList: Remove PackListImpl subclassHEADmasterIvan Frade10 days1-15/+5
| | | | | | | | | | PackList is an abstract class with two implementations, the default PackListImpl (never dirty) and NO_PACKS (always dirty). Simplify this to a class (PackList) with an special implementation for NO_PACKS. Change-Id: Ie97e1084417bbfa15c46bd570121a83883ed183c
* DfsObjDatabase.PackList: Remove dirty flag settersIvan Frade10 days1-34/+2
| | | | | | | | | | There are no callers for markDirty(), so in NO_PACKS dirty is always true and in the PackListImpl is always false. clearDirty() is a noop in both cases. Delete these methods to avoid confusions. Change-Id: I74d02be7dc287bef27fb7b9d6588c8d8c74105a8
* DfsPackFile: Delete #getIndexedObjectSize for ObjectIdIvan Frade12 days1-31/+1
| | | | | | | | | | | | | This method is inneficient, as callers lookup the object in the primary index twice: once to find it is in the pack (before calling), another for the position (in the method). Now we have \#getIndexedObjectSize(ctx, indexPosition) variant, so callers find the pack with \#findIdxPosition() and reuse the idxPosition here. Remove this variant with objectId. It doesn't do anything that the combination above cannot do. Change-Id: Ia913f511338b44ab5c45c55d47754d299946f03c
* MultiPackIndexLoader: Add NON-NLS annotation to messageIvan Frade12 days1-1/+1
| | | | | | Reported as warning in the mvn build. Change-Id: I981947c5717b74ef9547322b25a7211337b32cf6
* ChangedPathFilter: Suppress nls warnings in toStringIvan Frade12 days1-0/+1
| | | | | | | | | | mvn build was warning about these messages lacking the NLS annotation. As the method has 3 lines with 3 string, just suppress the warning in the whole method. Change-Id: I0dbd4055266517edd21418dc12528776078c34cc
* DfsReader: Reuse index position when looking up object sizeIvan Frade12 days2-4/+32
| | | | | | | | | | | | To get the indexed size of an object, we do two lookups in the primary index. First to find the object, then to find its position in the index. This hits performance when checking many object sizes on big packs. The first lookup of the object returns the index position. Reuse it for the object size index. Change-Id: If875d1b5d0b70f0cf4c448e0f9da09db65caac5e
* DfsReader: Use new findPack instead of findPackWithObjectIvan Frade12 days1-32/+13
| | | | | | | | | | Now findPack does the same than findPackWithObject, but it also returns the index position of the object, which we wil use in the next change. In getObjectSize and isNotLargerThan, use "findPack" to lookup the object. There are no more usages of findPackWithObject. Change-Id: I9d26a1bf4ac4e8f87148be90e25d17efa4b70e69
* DfsReader: return idx position when looking up object in packsIvan Frade12 days2-13/+30
| | | | | | | | | | | | | This is a preparatory change to avoid double lookup in the primary index when using the object size index. If we get the index position during the object lookup, we dont need to query the index again when looking up the size. Make "hasImpl" return the index position of the object in the pack (besides leaving "last" pointing to the pack). Move the "has" logic that checks "last" before other packs to a method we can reuse. Change-Id: I36d2b348971a642c54da3d3727c04d0134ca9eba
* DfsReader: Handle correctly invalid object-size indexesIvan Frade12 days1-13/+7
| | | | | | | | | | | | | | | | | safeGetIndexedObjectSize returns an Optional to differentiate between "object not found in index" and "index not found". This is wrong for two reasons: the IOException indicating "index not found" is only thrown once (on first loading failure) and the first loading happens in the "hasObjectSizeIndex" just before. Rethrow the IOException in getIndexesObjectSize as IllegalStateException (as it shouldn't happen) and return a plain long in the call. As a side, this saves instantiating an Optional per call and simplifies further optimizations. Change-Id: Ic728483f0e1ec7f98495fdbdb15dc0b24f30dbfd
* Merge branch 'stable-7.2' into stable-7.3stable-7.3Matthias Sohn2025-06-201-0/+1
|\ | | | | | | | | | | | | * stable-7.2: Fix: Close the "preserved" PackDirectory Change-Id: If9f2cd1278aad72f6ca7cae72fbd6b9f5ec66bc8
| * Merge branch 'stable-7.1' into stable-7.2Matthias Sohn2025-06-201-0/+1
| |\ | | | | | | | | | | | | | | | | | | * stable-7.1: Fix: Close the "preserved" PackDirectory Change-Id: I82f138f134fe09717e2e024b3c87971140f01b29
| | * Merge branch 'stable-7.0' into stable-7.1stable-7.1Matthias Sohn2025-06-201-0/+1
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: Fix: Close the "preserved" PackDirectory Change-Id: Icd3f79322f8c021e18fd5c881cd9f2a406230fa8
| | | * Merge branch 'stable-6.10' into stable-7.0stable-7.0Matthias Sohn2025-06-201-0/+1
| | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: Fix: Close the "preserved" PackDirectory Change-Id: Ie0ecfd8178ef4e2eef6a29d46be5645648fe88f3
| | | | * Fix: Close the "preserved" PackDirectorystable-6.10Nasser Grainawi2025-06-191-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has been missing since the feature was first added in commit 6167641834e28f8ad322f8fde60866b339bfb7fe. It's possible we could be more aggressive and close soon after attempting to get an object from the preserved packs, but for concurrent misses that might cause thrashing. More likely it would be safe to attempt closing after successfully restoring a preserved pack. A follow up change should attempt that. Change-Id: I87d61007bcc3d03fc86bd18465ca66a2e6f697a1
* | | | | Merge branch 'stable-7.2'Matthias Sohn2025-06-031-26/+32
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.2: Use the same ordering/locking in delete() as C git Change-Id: I0b583cd218c39b1dc0726ae82da86eca58cc81eb
| * | | | Merge branch 'stable-7.1' into stable-7.2Matthias Sohn2025-06-031-26/+32
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.1: Use the same ordering/locking in delete() as C git Change-Id: Id52c938b041604162dca9162726bfb594e96f5d1
| | * | | Merge branch 'stable-7.0' into stable-7.1Matthias Sohn2025-06-031-26/+32
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: Use the same ordering/locking in delete() as C git Change-Id: I2c38321ee410d9ec60481d56315710beaebd393a
| | | * | Merge branch 'stable-6.10' into stable-7.0Matthias Sohn2025-06-031-26/+32
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: Use the same ordering/locking in delete() as C git Change-Id: I0d06e39d06315e0b9e770bdf79164779d98f9f50
| | | | * Use the same ordering/locking in delete() as C gitDaniele Sassoli2025-05-291-26/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Following the examples of cgit, lock packed-refs *before* checking for existance of refs in it [1] and *keep the lock* until the loose ref (if any) is removed [2]. The packed-refs lock is kept even when no packed-refs update is required [3] so that somebody else doesn't pack a reference that we are trying to delete. This fixes a concurrency issue that happens on projects with a substantial amount of refs(>~500k) where packing takes long enough for a ref deletion to be triggered half way through it. Not locking the packed-refs file before checking if the refs exists is not safe, as it opens up situations where loose refs are repacked in memory and locked on disk, but before the lock is released and packed-refs is flushed to disk, a ref is deleted. As packed-refs was NOT locked while checking wether a ref existed in it, the current content on disk was read, which was about to be overwritten and did not contain the ref about to be deleted. As the delete doesn't see the ref in the current, on-disk, version of packed refs, it skips processing altogether and moves on, correctly, deleting only the associated loose ref and leaving the packed one behind. Once the new packed-refs, containing the ref that was just deleted, was commited to disk, the ref would come back to life. Therefore, the packed-refs needs to be locked before checking if it contains a ref or not in the same way the C implementation of Git does at [1]. There are tradeoffs, though, in this decision, which will reduce the parallelism of deleting loose refs and performing the refs repacking, which happens very often in certain JGit implementations like Gerrit Code Review. Before this change, repacking of refs and removal of loose refs unrelated to the in-flight repacking was possible without involving any locking; after this change, all loose refs removals have to wait for the packing of refs to be completed, even though the repacking and the refs removals were completely unrelated and their namespaces disjoint. See more details on the test's performance results and the associated tradeoffs in the Issue jgit-152. NOTE: This delete ref locking logic was incorrect regardless of how the packing of the refs is implemented. Making decisions if the pack transaction is needed or not on an unlocked resource is racy and also flagged as bug at [1]. [1]https://github.com/git/git/blob/master/refs/packed-backend.c#L1590 [2]https://github.com/git/git/blob/master/refs/files-backend.c#L3261 [3]https://github.com/git/git/blob/master/refs/files-backend.c#L2943 Bug: jgit-152 Change-Id: I158ec837904617c5fdf667e295ae667b2f037945
| | * | | Merge branch 'stable-7.0' into stable-7.1Matthias Sohn2025-05-222-1/+20
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: Prepare 7.0.2-SNAPSHOT builds JGit v7.0.1.202505221510-r Prepare 6.10.2-SNAPSHOT builds JGit v6.10.1.202505221210-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities Change-Id: I4506e4bf51225000418b15bf09df3287be26242a
| | | * | Merge branch 'stable-6.10' into stable-7.0Matthias Sohn2025-05-222-1/+20
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: Prepare 6.10.2-SNAPSHOT builds JGit v6.10.1.202505221210-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities Change-Id: I699d57974d9ef2428355c59194c6becbc16828b7
| | | | * AmazonS3: Do not accept DOCTYPE and entitiesMatthias Sohn2025-05-221-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This follows OWASP recommendations in https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html Change-Id: I3d47debf14d95c8189d51256b4eb2ba991279452
| | | | * ManifestParser: Do not accept DOCTYPE and entitiesIvan Frade2025-05-221-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These open the door for XXE attacks [1] and manifest do not need them. [1] https://en.wikipedia.org/wiki/XML_external_entity_attack Change-Id: Ia79971e1c34afaf287584ae4a7f71baebcb48b6a
* | | | | FS.getFileStoreAttributes: cancel failed task executed asynchronouslyMatthias Sohn2025-05-211-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to avoid unnecessary computations if the task getting FileStoreAttributes asynchronously timed out, cancelled or failed with another exceptions. Change-Id: I127a3e2f3710fc5a8742c61f513576ee2d84baed
* | | | | DfsReader/PackFile: Move represention to the packfileIvan Frade2025-05-203-39/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The reader iterates the packs looking for the best representation of an object. It does so using the primary and reverse indexes of the pack. This direct use of the indexes prevent the pack to be more clever and use the multipack index in these operations. Move finding objects in the pack and the representation to the pack itself. Now the pack can decide how to implement it (e.g. using the multipack index). Change-Id: Ief0f2384ab395557c2533990f7b3c532a88d7ac9
* | | | | DfsPackFile: Remove unused getObjectCount methodIvan Frade2025-05-191-14/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This method doesn't seem to be used. Change-Id: I9b7a25aa7bd99b461bb79a0d25c61a1fd6d40e64
* | | | | Merge branch 'stable-7.2'Matthias Sohn2025-05-152-1/+20
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.2: Prepare 7.2.2-SNAPSHOT builds JGit v7.2.1.202505142326-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities Change-Id: I2a56d92735c003d7d61147743409a24e1539ad55
| * | | | AmazonS3: Do not accept DOCTYPE and entitiesMatthias Sohn2025-05-151-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This follows OWASP recommendations in https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html Change-Id: I3d47debf14d95c8189d51256b4eb2ba991279452
| * | | | ManifestParser: Do not accept DOCTYPE and entitiesIvan Frade2025-05-151-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These open the door for XXE attacks [1] and manifest do not need them. [1] https://en.wikipedia.org/wiki/XML_external_entity_attack Change-Id: Ia79971e1c34afaf287584ae4a7f71baebcb48b6a
* | | | | FileReftableStack: ensure new reftable files aren't missed on NFSMatthias Sohn2025-05-091-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If "core.trustTablesListStat" is set to "after_open" to ensure file attributes are refreshed on NFS refreshing only the attributes of the tables.list file is not sufficient since then NFS may not detect new reftable files (stored in the same refs/reftable/ directory) which may cause FileNotFoundExceptions. Fix this by refreshing attributes of the refs/reftable/ directory. Change-Id: I7e07834fd7628a07ed644cb2740101a749d433bb
* | | | | Encapsulate layout of reftable stack in FileReftableStackMatthias Sohn2025-05-092-44/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The filesystem layout of reftables is fixed in the reftable specification hence the constructor FileReftableStack(File tablesListFile, File reftableDir, @Nullable Runnable onChange, Supplier<Config> configSupplier) should not allow to pass the path of the tables.list file independently from the refs/reftable/ directory containing both the tables.list file and the reftable files. Hence remove the path of the tables.list file from its argument list and instead set it inside the constructor. Use reftableDir instead of tablesListFile.getParentFile(). Also rename FileReftableStack.stackPath to tablesListFile which is easier to understand and remove the useless return value of FileReftableDatabase#convertFrom which was unused and always returned null. Change-Id: If1efba5d49d979d99a86871996abe550422d8945
* | | | | Merge "PlotRefComparator: fix #timeof"Matthias Sohn2025-05-071-2/+3
|\ \ \ \ \
| * | | | | PlotRefComparator: fix #timeofMatthias Sohn2025-05-061-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The #timeof method returned - the commit time in seconds since the epoch for RevCommits - the tagger time in milliseconds since the epoch for annotated tags Fix this by converting commit time to milliseconds to ensure consistent timestamp comparison of tags and commits. Change-Id: I91621da19c234fe823788d85e4077750cb5cba1a
* | | | | | Merge "DfsInserter: Remove checkExisting setter and parameter"Ivan Frade2025-05-061-14/+2
|\ \ \ \ \ \ | |/ / / / / |/| | | | |
| * | | | | DfsInserter: Remove checkExisting setter and parameterIvan Frade2025-05-061-14/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We turned this off to debug something else and it created significantly bigger packs that spammed the storage. There is no reason to allow duplicates while inserting in a pack, so lets remove completely the option. The setter is not used and the variable is always true. Change-Id: I978049dce30c33b7aa650d3ddef9430cbef764fa
* | | | | | PlotWalk: Replace call to deprecated method in PersonIdentIvan Frade2025-05-061-1/+1
|/ / / / / | | | | | | | | | | | | | | | Change-Id: I0972121aaba6fd05f733d3ec62515e00c30530c7
* | | | | midx: Fix license header from Google Inc. to LLCIvan Frade2025-05-024-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This code was introduced recently and the license header says Google Inc. where it should be Google LLC. Change-Id: If3d4f992bdcd165cfba356b4f9721c4605017dda
* | | | | PackExt: Add value for the multipack indexIvan Frade2025-04-152-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code establishing the PackExt from the file name uses endsWith, i think to cover for the x.<ext> and x.old-<ext> cases. This doesn't work if we add the "midx" case, as "midx".endsWith("idx"). Check explicitely that the extension maches itself or with the old- prefix. We rather keep the "midx" as file extension for upstream compatibility. Change-Id: I25d0e850b9df17ed99d776b397560774bc402d39
* | | | | MultiPackIndex: Add and implement #resolve() methodIvan Frade2025-04-152-1/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The reader offers the resolve functionality to find object ids with certain prefix. The basic implementation iterates through packs and calls resolve in their indexes. With multipack index, we can answer the resolve directly for all the packs included. Offer a resolve() method and implement it in the multipack index. Change-Id: If5679652f149a41afe568c719ba40b291ae1b917
* | | | | MultiPackIndexWriter: Handle empty packsIvan Frade2025-04-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a pack doesn't have data, its offsets are null, not empty. A pack without objects is probably a pathological case, but I bumped into this while writing other tests. Check if the offsets are null (instead of empty) before trying to write the ridx. Change-Id: I8cadea086b302b2ead02a5a8d4e9e8adf85bc07b
* | | | | MultiPackIndexWriter: return bytes writtenIvan Frade2025-04-141-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is useful later, in GC, to write the size of the stream in the pack description. Change-Id: Ic3d759c1120ef2fde77b432d33c46ba1791d954b
* | | | | MultiPackIndexV1: Calculate memsize with longIvan Frade2025-04-111-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Errorprone pointed out that adding with int and casting to long at the end could overflow. Mark one of the operands as long to do the addition directly as long and avoid the overflow risk. Change-Id: I12449af90c84044a04e6dbd0978ff48d0ea33674
* | | | | MultiPackIndexV1: Reorder methods and add NON-NLS annotationsIvan Frade2025-04-091-13/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cosmetic changes. Change-Id: I5f4d463cbb4e256d068369efc798c1c00f17c18c
* | | | | MultiPackIndex: add #getMemorySize() methodIvan Frade2025-04-092-1/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We need a size estimation to store this ref in the DfsBlockCache. Change-Id: I974099cf66a66d09320a795638abfeac506da2c1
* | | | | midx: reader for the multipack indexFabio Ponciroli2025-04-085-0/+651
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reader and java API to load and query a multipack index. Note: the `multi-pack-index.v1` file used in the tests has been created on a local repository with the "git multi-pack-index" command [1]. [1]: https://git-scm.com/docs/git-multi-pack-index Change-Id: I09def882e1e77e9f22c6236b0a035a1e80bc1ace
* | | | | DfsPackFile.PackIndexes: pass primary index to reverse indexIvan Frade2025-04-071-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | DfsPackFile.PackIndexes interface doesn't cache anything and assumes that the caller will keep the loaded references. When the DFS cache implementation needs a primary index to create the reverse index, it invokes its own getIndex method, skipping the reference hold by the caller and potentially triggering reloads. Pass the primary index as parameter in #getReverseIndex(), so the caller can reuse its reference. Change-Id: I1f310eecea73d8f1afc1fb94d40cd94cccdb5996
* | | | | Merge "SystemReader: Add support for XDG_CACHE_HOME"Ivan Frade2025-03-213-0/+41
|\ \ \ \ \
| * | | | | SystemReader: Add support for XDG_CACHE_HOMEIvan Frade2025-03-213-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In following changes we introduce a cache for the Blame CLI and it should follow the XDG standard for the location. Add support for XDG_CACHE_HOME following the XDG_CONFIG_HOME pattern. Change-Id: I622f7eb7ff942fafdb5c5da877d1fb1507d5e482
* | | | | | Removed redundant TreeFilter.ANY_DIFF in LogCommand.youtirsin2025-03-201-3/+3
|/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I2ecf5f6c220e88dcc3ec2bf132e71156cf0fe622 Signed-off-by: kylezhao <kylezhao@tencent.com> Signed-off-by: youtirsin <brocodzhang@tencent.com>
* | | | | BlameResult: Let generator decide when to use the blame cacheIvan Frade2025-03-171-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BlameResult disables the blame cache in the generator, just in case the generator is setup for a reverse blame (where we cannot use the cache). This prevents the use of the cache by defaut in the Blame cli. The generator already takes care of disabling the cache in reverse blames, so this is unnecessary. Remove the forced disable and let the generator decide when it can use the cache. Change-Id: I451b46c3b88c83025276150253a5396ea59b9f54
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-07 01:02:42 +0000