« Berkeley DB Java Edition: Why Database.preload() doesn't always help | Main | Berkeley DB Java Edition 4.0 Released »

Berkeley DB Java Edition vs Windows 7 IO

In a previous post I raised the topic of IO problems on Windows 7 (fsync was resulting in "Incorrect Function" IOExceptions). This again showed up more recently in this OTN thread. Fortunately, the reporting user supplied us with a reproducible test case which allowed us to characterize the problem.

At this point I am reasonably certain that the problem has to do with a write() call being initiated on a RandomAccessFile when an fsync() is already in progress in another thread (i.e. a concurrent fsync and write on the same file, but not with the same file descriptors). JE routinely performs concurrent IO operations on a given file. In the particular test case, it is by virtue of the checkpointer initiating an fsync while the user application thread is writing.

It turns out that in ext3 we previously encountered a performance slowdown because that file system takes an exclusive mutex on the inode for any IO operation, and therefore an fsync will block reads and writes. JE 4.0 has a "fix" to this problem which is described here. While the 4.0 "write queue" work improves performance on file systems like ext3 which take exclusive mutexes on the inode for any IO operation, it also has the added benefit that JE no longer performs concurrent fsync and write operations.

That said, there seems to be a true bug in Windows 7 IO, if for no other reason than I observe corruption on sector boundaries in the log files which are produced by the test case (JE does no operations on sector boundaries).

Comments (4)

Hey Charles,

Another reason to get JE 4.0 out quickly. ;)

Best,
Ismael

Bertold Kolics:

Great post. Is there a file system you would recommend using with BDB JE on Linux?

Hello Bertold,

We don't have a preference because I have a suspicion that all the fs's on linux grab a mutex on the inode during any I/O operation. So I think the most important thing is to use JE 4.0 which has the write queue code incorporated into it.

-cwl

r4ds:

Hi..
I think, there is not any kinds of comparison between the Java DB and Window7 I/O because the Window 7 is an operation system and Java is the Platform independent environments. It might be another reason to get JE 4.0 out quickly...

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

About This Entry

This page contains a single entry from the blog posted on October 27, 2009 10:00 PM.

The previous post in this blog was Berkeley DB Java Edition: Why Database.preload() doesn't always help.

The next post in this blog is Berkeley DB Java Edition 4.0 Released.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type and Oracle