In a previous post I raised the topic of IO problems on Windows 7 (fsync was resulting in "Incorrect Function" IOExceptions). This again showed up more recently in this OTN thread. Fortunately, the reporting user supplied us with a reproducible test case which allowed us to characterize the problem.
At this point I am reasonably certain that the problem has to do with a write() call being initiated on a RandomAccessFile when an fsync() is already in progress in another thread (i.e. a concurrent fsync and write on the same file, but not with the same file descriptors). JE routinely performs concurrent IO operations on a given file. In the particular test case, it is by virtue of the checkpointer initiating an fsync while the user application thread is writing.
It turns out that in ext3 we previously encountered a performance slowdown because that file system takes an exclusive mutex on the inode for any IO operation, and therefore an fsync will block reads and writes. JE 4.0 has a "fix" to this problem which is described here. While the 4.0 "write queue" work improves performance on file systems like ext3 which take exclusive mutexes on the inode for any IO operation, it also has the added benefit that JE no longer performs concurrent fsync and write operations.
That said, there seems to be a true bug in Windows 7 IO, if for no other reason than I observe corruption on sector boundaries in the log files which are produced by the test case (JE does no operations on sector boundaries).
Comments (4)
Hey Charles,
Another reason to get JE 4.0 out quickly. ;)
Best,
Ismael
Posted by Ismael Juma | October 27, 2009 10:38 PM
Posted on October 27, 2009 22:38
Great post. Is there a file system you would recommend using with BDB JE on Linux?
Posted by Bertold Kolics | October 29, 2009 9:17 AM
Posted on October 29, 2009 09:17
Hello Bertold,
We don't have a preference because I have a suspicion that all the fs's on linux grab a mutex on the inode during any I/O operation. So I think the most important thing is to use JE 4.0 which has the write queue code incorporated into it.
-cwl
Posted by Charles Lamb | October 29, 2009 12:25 PM
Posted on October 29, 2009 12:25
Hi..
I think, there is not any kinds of comparison between the Java DB and Window7 I/O because the Window 7 is an operation system and Java is the Platform independent environments. It might be another reason to get JE 4.0 out quickly...
Posted by r4ds | November 8, 2009 11:55 PM
Posted on November 8, 2009 23:55