Understanding Compressed References
By tomas.nilsson on Oct 17, 2010
This post is a guest post from David Buck, our man in Tokyo. David (Well, everyone that knows him calls him Buck, but until that time you will have to call him David) has offered to write a few guest posts on deep JVM issues, and this is the first one.
Compressed references are an optimization that helps give 64-bit versions of JRockit a considerable speed boost (sometimes as much as 15% for memory intensive workloads). In this post, we will talk about what they are, how they work, and some of the issues you might run into when compressed references are enabled.
'Because of Java's Write Once, Run Anywhere design, migrating from a 32-bit to a 64-bit JVM is usually painless as your Java code can be left untouched (as long as it doesn't rely on JNI). By simply swapping JVMs, you gain the advantages of running as a 64-bit process (huge Java heap sizes, more CPU registers, etc) without even having to recompile your Java code. This is exactly what the Java platform has always promised: allowing applications to take advantage of new hardware and advances in virtual machine technology without the need to update the application itself.
However, there are several tradeoffs (usually performance related) that must be considered when choosing between a 32-bit and 64-bit JVM. In some circumstances, the increased code size and pointer length of a 64-bit process can result in less cache efficiency and heaver demands on CPU memory bandwidth. While each application is different and needs to be load-tested to find what JVM will give you the best performance, a 32-bit JVM will often run the same Java application faster than its 64-bit counterpart.
Compressed references represent a solution that allows your application running on a 64-bit JRockit JVM to enjoy some of the performance benefits of a 32-bit JVM while retaining the advantages of a 64-bit JVM. When you run with a Java heap size that is small enough, compressed references will allow use of 32-bit object references (pointers) on a 64-bit JRockit JVM. Use of 32-bit pointers not only allows for much faster performance, it also reduces the amount of heap memory required to store any Java object that references other objects.
Fortunately, you do not need to understand the details of how compressed references work to benefit from the performance advantages they provide. Anyone out there who just wants the need-to-know information, please feel free to jump to the "Usage" section below. For those of you who think compressed references sound too-good-to-be-true and love details, please read on!
How do we use 32-bit pointers with a 64-bit address space? Well, some behind- the-scenes trickery is involved to give your applications the best of both worlds.
There are 3 types of compressed references and each one corresponds to maximum heap size supported: 4GB, 32GB and 64GB.
The easiest scenario to understand is the use of 4GB compressed references where the Java heap is small enough that we can store the entire heap in the first 4GB of address space. With the heap within the 32-bit addressable range, we can simply use a 32-bit pointer as-is to point to Java objects on the heap. Because this is the simplest form of compressed references to implement, it is also the oldest. JRockit has featured 4GB compressed references for many years now.
Once you cross the 4GB barrier, things get a little bit more complicated. We can no longer directly use a 32-pointer to reference all of the objects in the heap. We are helped by a fortunate aspect of JRockit's internal design: Every single object on the heap is aligned to an 8 byte boundary. In other words, every single object address will always be a multiple of 8. Those who are used to bit fiddling will quickly realize that the last 3 bits of any such address will always be 0. In other words, we are only "using" the higher order 29 bits of each 32-bit heap address.
The key to making use of these "unused" bits is to rotate the each address by 3 bits to the right before storing each heap address, and then rotate them back 3 places left later when we need to de-reference them. This allows us to address a 35-bit address space (35GB) with a 32-bit pointer. This may sound complicated, but a rotation like this is executed in a single machine code instruction and the performance impact of rotating the address for each access is close to negligible (when compared to the simpler under-4GB compressed reference, we've seen at most a 2% performance hit). These types of compressed references are known as 32GB compressed references.
Every year, the demands applications place on the JVM for memory and performance increase dramatically. Especially for J2EE server side application, the use of larger and more complicated frameworks combined with ever more ambitious projects require performance and functionality unimaginable only a few years ago. Case in point: we have more customer than ever who are using larger than 32GB Java heaps. For these customers, we have one last type of compressed reference: 64GB.
For heap sizes between 32GB and 64GB, a variation on the bit-rotation trick described above is used. With 64GB compressed references, JRockit will limit itself to only storing objects on 16 byte boundaries. For the price we pay in the form of a very small increase in fragmentation (the wasted space between objects that don't each consume an exact multiple of 16 bytes), we now have 4 bits for each 32-bit address that are guaranteed to be 0 and therefore can be shifted to give us a 36-bit (64GB) range.
The point is that compressed references use every single bit of a 32-bit value to reference as much address space as possible. By making a few assumptions about where we store the Java heap (within the 64-bit address space) and how we store the addresses that point to it, we can avoid the overhead of using 64-bit pointers even though we are running on a 64-bit JVM.
For most users, compressed references work perfectly right out of the box. In fact, you may be using them already and not know about it! Depending on the maximum heap size (-Xmx) specified on the Java command line, JRockit will automatically enable the use of compressed references. With JRockit R28, the following defaults are used.
-Xmx: <= 3GB -- 4GB compressed references
-Xmx: <= 25GB -- 32GB compressed references
-Xmx: <= 57GB -- 64GB compressed references
Earlier versions of JRockit (R27 and earlier) only supported 4GB compressed references. If you are using a larger heap size (still under 64GB), you may want to consider upgrading to JRockit R28 so you can gain the advantage of compressed references. From JRockit R26.4, compressed references are enabled by default for all heap sizes below 4GB.
While JRockit's out-of-the-box behavior is ideal for most users, you can disable compressed references or even specify the type of compressed reference used (R28 only) For more information on overriding JRockit's default behavior, see the on-line documentation:
Occasionally the use of compressed references can cause problems if the JVM runs low on available address space below the 4GB barrier. There are other data structures and code that can only be stored in the first 4GB of address space (usually because they are also referenced by 32-bit pointers at some point). When using 4GB compressed references, the entire Java heap is also stored in this valuable under-4GB address range, possibly leading to native OutOfMemoryErrors (OOME). In practice, 3GB will be close to the largest heap size you'll want to use with 4GB compress references.
Because of the practically limitless address space that can be reserved by a 64-bit process, native OOMEs are very rare on 64-bit platforms. If you are seeing a native OOME with a 64-bit JRockit and are using 4GB compressed references, exhaustion of the below-4GB address space is most likely the cause of your unhappiness.
Luckily this problem can be fixed very easily. On more recent JRockit releases (R28), manually forcing JRockit to use 32GB compressed references (as opposed to 4GB compressed references) will allow the heap to be stored above the 4GB range leaving ample room for any other data that needs to be stored in this valuable address space.
$java -Xmx3584m -XXcompressedRefs:size=32GB MyWonderfullJavaApp
On older releases (R27 and earlier), only 4GB compressed references were supported, so the only option is to disable compressed references:
$java -Xmx3584m -XXcompressedRefs=0 MyWonderfullJavaApp
Again, R26/R27 users who have heap sizes too big for 4GB compressed references will want to consider upgrading to R28 take advantage of the added compressed reference support for larger heap sizes.
Compressed references can really improve the performance of applications that run on 64-bit JRockit. For many applications, it can smooth the transition from a 32-bit JVM by fixing what would otherwise be a substantial performance hit. Even better, they are simple to use and work right out of the box. In fact, you may already be benefiting from them already!
PS. The HotSpot JVM also now supports a similar feature known as CompressedOops.