The JE DbCacheSize Utility
Often users experience performance problems related to improper cache sizing. A useful tool, albeit an undocumented one, for helping to size caches is com.sleepycat.je.util.DbCacheSize. The header comments in the source code provide a good description of how to invoke it, what it calculates, and what its limitations are. Briefly, you provide (at a minimum) the key size and the number of records. Optionally you can also provide data size, "overhead" percentage, btree fanout, node density, and a directory in which to perform a trial run. The program then provides an estimate of the minimum and maximum cache size required for holding the database in memory. This comes in two forms: (1) the cache size to hold the internal nodes, and (2) the cache size to hold the internal and leaf (i.e. data) nodes. To account for various overheads, an "overhead" percentage is added. So the output has both "Cache Size" and "Btree Size", the former including the overhead and being the larger.
Here is an example run (again, taken right from the source code class header comments):
$ java DbCacheSize -records 554719 -key 16 -data 100
Inputs: records=554719 keySize=16 dataSize=100 nodeMax=128 density=80%
overhead=10%
Cache Size Btree Size Description
------------ ------------ -----------
30,547,440 27,492,696 Minimum, internal nodes only
41,460,720 37,314,648 Maximum, internal nodes only
114,371,644 102,934,480 Minimum, internal nodes and leaf nodes
125,284,924 112,756,432 Maximum, internal nodes and leaf nodes
Btree levels: 3
This says that the minimum cache size to hold only the internal nodes of the btree in cache is approximately 30MB. The maximum size to hold the entire database in cache, both internal nodes and datarecords, is 125Mb.
