Charles Lamb's Blog

[This is a repost of the 11/30/05 Sleepycat blog entry.]

A user recently noted that while running JE in a 64b JVM
OutOfMemoryErrors happened, whereas they didn't happen under a 32b JVM.
The reason was apparent. JE tries hard to only use as much memory as
the application program has given it in cache (e.g. using the je.maxMemory
parameter). How can a Java library like JE know just how much memory is
being used by the objects that it's responsible for? It has to keep a
running tally of each object in the JE cache.

But how does it
figure out the size of each object? One answer is to use the Java 5
reflection APIs to compute the size of an object at run time.
Unfortunately, that would be too slow and also would only be available
on Java 5 (JE is supported on 1.4 JVMs). Instead, for each object that
can use space in the cache (and some others that aren't in the cache,
like Transactions), we take a constant value as the base amount of
memory used -- we calculate this at "build" time -- and add to it the
size of variable components (e.g. arrays that reference other objects,
HashSets and their elements, etc.)

To determine the size of an element we use a "hack" Sizeof program, similar to the one here.
Not surprisingly, when we ran our Sizeof program in a 64b JVM, the
results were quite a bit different. Many of them were 50% different and
that means that the cache usage calculations for JE 2.0.x are quite
wrong under a 64b JVM.

JE 2.1 will have a fix that will
determine at runtime what JVM architecture JE is running on and then
use an appropriate set of constants. Until then, if you run on a 64b
JVM, give about 50% more heap than you would normally (but leave your
cache size the same).