Berkeley DB Java Edition on ZFS Tuning Note
By Charles Lamb on Aug 24, 2010
I have been spending some time working on tuning continuous write load tuning on a Solaris 10/ZFS based system. One of the issues I observed is a drop in throughput every several seconds. Removing the fsync() calls from JE (you would never want to do this normally) smoothed out the dips in throughput which pointed to IO issues.
My colleague Sam pointed me at this discuss-zfs entry. And indeed, adding
set zfs:zfs_write_limit_override = 0x2000000
to /etc/system (with the fsync() calls put back into JE) does in fact seem to smooth things out (0x2000000 = 33MB). Of course, you'll want to adjust that parameter based on what size of on-board disk cache you have.
One way to get a rough indication of whether this is a potential problem is to use iostat and see if there are IO spikes.