You wouldn’t hammer a screw or screw a nail, you would use the
right tool for the job. That’s the elegance of the ZFS Storage Appliance, it simply
uses the right tools for the jobs it needs to do.
The Hybrid Storage
Pool within the ZFS Storage Appliance has three different caches, using three
different technologies to perform three different jobs. I’ll explain each in a
little detail below:
The Write Cache uses
write optimised SSDs (aka Logzillas). These get hammered (excuse the pun) and therefore
needs to have high levels of endurance and low latency write response. They don’t
require large capacity (only a few GB are ever used at any given time) and they
are almost never read. As such, we use a specific type of SSD for this job. The
benefit the write cache provides is twofold:
The 2nd cache is the Adaptive Replacement Cache (ARC).
The ARC is held in DRAM which in a ZFS Storage Appliance is comparatively
pretty large (up to 3TB on a ZS4-4 controller pair). DRAM of course is the
fastest form of storage. It serves up I/O measured in nanoseconds compared to
the much slower microseconds of flash drives and milliseconds of hard drives.
We consider ZFS to be a DRAM first storage architecture and it is one key part
of the blinding performance ZFS can deliver.
The benefit of the ARC is threefold:
The 3rd cache is the Layer 2 Adaptive Replacement Cache (L2ARC) which uses large read optimised
SSDs (aka Readzillas) which have large capacity (up to 12.8TB per controller
pair) and great read performance, with more modest write and endurance
characteristics. The role of the L2ARC is to collect smaller pieces of data
that fall out of the ARC. This typically consists of data that is <32K in
size and has a reasonable level of re-use. The L2ARC is like a bench player, typically
not receiving much fanfare, but invaluable when called into action. The benefit
of the L2ARC is threefold:
Using these three different technologies to act as three
different forms of cache has the effect that a final technology (being the
HDDs) is left to do what it does well. That is, storing large qualities of data
in an efficient manner using large physical I/O, therefore making the most of
every single rotation of the spindle.
Pulling this all together you not only get the fastest
storage, but also the most efficient, by simply using the right tool for the