While working on performance measurement and improvements for JE High Availability (HA) we noticed that we could eliminate a buffer copy by using a gathering write. The changes were simple enough but when I made them, weird things started happening with the stress test I was using. It seemed like the ByteBuffers which were supposedly being sent from master to replica were ending up corrupted over when they got to the replica side. Eventually, we decided that bug 6812202 was responsible (even though we didn't always see an OOME) and in fact, moving to the JDK 1.7 "fixes" the problem. Unfortunately, we are not in a position to pre-req Java 7 for HA or JE for that matter.
You're probably wondering if this means that HA is imminent. Yes, the beta is expected to start in a few weeks, which is a very exciting development for JE. And you may be wondering what we know about HA performance. I'll just say that we're quite comfortable with where HA performance is right now. We know of some areas where we can make improvements and we'll eventually make them, but for now, we like what we see.
Comments (3)
I filed bug 6812202 and also exchanged some e-mails with sun concerning "gathering write".
You say you "could eliminate a buffer copy by using a gathering write" - but you will only eliminate the copy if your are already operating on direct buffers. If not the copy will occur in the implementation of the gathering write (in jdk7 since b50) In earlier versions you may get the OOM - or the size of the process (not heap memory) will blow up.
Regards, Peter
Posted by jodeleit | June 5, 2009 6:54 AM
Posted on June 5, 2009 06:54
Update: A quote from Alan Bateman and Jeanfrancois Arcand from their talk "Asynchronous I/O Tricks
and Tips" on JavaOne this year
"Scattered ByteBuffer write operations still offer you
free copy, using direct ByteBuffer or not!"
(on slide 40, http://www28.cplan.com/cb_export/PS_TS-4222_304222_230-1_FIN_v1.pdf, login is contentbuilder/doc789)
And don't be misguided by the word "free".
Posted by Peter Jodeleit | June 6, 2009 1:11 PM
Posted on June 6, 2009 13:11
Thanks for the pointers Peter. I should point out that even though the gathering writes may still be copying, the work we were doing was to eliminate a copy in our own code. So the net result would still be one less copy.
Posted by Charles Lamb | June 8, 2009 8:53 AM
Posted on June 8, 2009 08:53