]> source.dussan.org Git - jgit.git/commit
Fix concurrent creation of fan-out object directories 72/9372/2
authorRoberto Tyley <roberto.tyley@gmail.com>
Tue, 25 Dec 2012 09:10:51 +0000 (09:10 +0000)
committerGerrit Code Review @ Eclipse.org <gerrit@eclipse.org>
Sun, 13 Jan 2013 23:59:59 +0000 (18:59 -0500)
commit5dcc8693d7d1ab81e3d6e6583a3918a3b6003bad
tree3ead984ff904d1c0d6a9c26a51b9c4b19f2edfd4
parent912ef3da19c7bc1975805e1e3e9746baf479c2be
Fix concurrent creation of fan-out object directories

If multiple threads attempted to insert loose objects into the same new
fan-out directory, the creation of that directory was subject to a race
condition that could lead to an unnecessary IOException being thrown -
because an inserter could not 'create' a directory that had just been
generated by a different thread. All we require is that the directory
does indeed *exist*, so not being able to _create_ it is not actually a
fatal problem. Setting 'skipExisting' to 'true' on the call to mkdir()
fixes the issue.

I found this issue as a real world occurrence while working on The BFG
Repo Cleaner (https://github.com/rtyley/bfg-repo-cleaner), a tool which
concurrently performs a lot of object creation.

In order to demonstrate the problem here I've added a small test case
which reliably reproduces the issue on the few different hardware
systems I've tried. The error thrown when the race-condition arises is
this:

java.io.IOException: Creating directory /home/roberto/repo.git/objects/e6 failed
at org.eclipse.jgit.util.FileUtils.mkdir(FileUtils.java:182)
at org.eclipse.jgit.storage.file.ObjectDirectory.insertUnpackedObject(ObjectDirectory.java:590)
at org.eclipse.jgit.storage.file.ObjectDirectoryInserter.insertOneObject(ObjectDirectoryInserter.java:113)
at org.eclipse.jgit.storage.file.ObjectDirectoryInserter.insert(ObjectDirectoryInserter.java:91)
at org.eclipse.jgit.lib.ObjectInserter.insert(ObjectInserter.java:329)

Change-Id: I88eac49bc600c56ba9ad290e6133d8a7113125ab
org.eclipse.jgit.test/tst/org/eclipse/jgit/storage/file/ObjectDirectoryTest.java [new file with mode: 0644]
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/ObjectDirectory.java