Gigaspaces curbs latency outliers with Java Real Time
By Frederic Pariente on Dec 19, 2008
"Latency matters. Amazon found every 100ms of latency cost them 1% in sales. Google found an extra .5 seconds in search page generation time dropped traffic by 20%. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition."
Latency is Everywhere (...), High Scalability Blog, Todd Hoff
Sun customers can turn to Gigaspaces to hide the network latencies and dramatically increase the performance of their multi-tier applications. Since the release of Java Real Time System (JRTS) 2.0, Gigaspaces customers can now turn back to Sun to further decrease the application latency of their business-critical transactions. Today's proofpoint is covering some engineering work at Gigaspaces this year around JRTS 2.0 to meet the stringent needs of our Financial Services customers with respect to market transaction latencies.
Today, most banks have migrated their internal software development from C/C++ to the Java language because of well-known advantages in development productivity (Java Platform), robustness & reliability (Garbage Collector) and platform independence (Java Bytecode). They may even have gotten better throughput performance through the use of standard architectures and application servers (Java Enterprise Edition). Among the few banking applications that have not been able to benefit yet from the Java revolution, you find the latency-critical applications connected to the trading floor. Why? Because of the unpredictable pauses introduced by the garbage collector which result in significant jitter (variance of execution time).
"If you've got some trades going through at 10 milliseconds and some at 1 millisecond, that's a problem. Our customers don't like variance."
Steve Rubinow, CTO, NYSE
In the context of a customer proof-of-concept this summer and in the light of the 2.0 release of Java Real Time System --JRTS 1.0 had the bad prerequisite of source code changes--, Gigaspaces revisited the opportunity for Java Real Time to serve the low-latency requirements of trading applications. Gigaspaces XAP 6.5, Solaris 10 and both Java 5.0 standard and real-time JVMs were used for the benchmark. The test scenario included a trade matching engine and multiple clients injecting messages at extreme speed. The success criteria was to get guaranteed latency per message under 10 msec, with no code modification to the matching engine. Which was achieved.
The first lesson learned was that msec latency was achievable with the standard JVM, through some advanced tuning of the JVM command-line options. While the customer had reported application freezes up to 20 sec during garbage collection under heavy load --he was running the JVM with no particular flags, unfortunately default JVM options optimize for throughput--, latencies could be brought down to milliseconds by switching to the Concurrent Garbarge Collector and applying these JVM flags :
-Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:CMSIncrementalDutyCycleMin=10 -XX:CMSIncrementalDutyCycle=50 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -Xmn150m -XX:MaxGCPauseMillis=2000 -XX:GCTimeRatio=10 -XX:+DisableExplicitGC
The second lesson learned was that the number of outliers can be reduced by an order of magnitude by using the real-time JVM. At a small cost in terms of application throughput --lower-- and CPU usage --higher-- of course. That behavior was highlighted in a related Gigaspaces scalability benchmark. Figures 1 and 2 below show a much more steady --read, less variance-- behavior of the application when run under Java Real Time System.
Figure 1 - Running standard JVM
Figure 2 - Running real-time JVM
The customer proof-of-concept stopped there but JRTS brings promises of sub-msec latencies through the use of a specific API --down to 20 microseconds for no-heap real-time threads! Check out this QCon London 2008 presentation on Java Real Time for latency-critical banking applications for coding details and use cases.
After Reuters in February, this is the second successful adoption of Java Real Time in Market Finance that I am aware of this year. I believe that we are seeing the beginning of the commercial take-off for Java Real Time on Wall Street. Not only it released in a production-ready version with JRTS 2.0 this year, but the Java Real Time revolution continues as presented by Sun and IBM during JavaOne 2008. While JRTS is available on Linux as of release 2.1, I anticipate that banks would target development then deployment on Solaris x64 --dtrace is probably the best tool to help fine-tune latency-critical applications--, a platform that has gained momentum in Financial Services since the release of Solaris 10, as exemplified by the recent record benchmark with Reuters Market Data System.