An Oracle blog about ZFS Storage

  • January 19, 2015

Using the Right Tool for the Job

You wouldn’t hammer a screw or screw a nail, you would use the
right tool for the job. That’s the elegance of the ZFS Storage Appliance, it simply
uses the right tools for the jobs it needs to do.

The Hybrid Storage
within the ZFS Storage Appliance has three different caches, using three
different technologies to perform three different jobs. I’ll explain each in a
little detail below:

The Write Cache uses
write optimised SSDs (aka Logzillas). These get hammered (excuse the pun) and therefore
needs to have high levels of endurance and low latency write response. They don’t
require large capacity (only a few GB are ever used at any given time) and they
are almost never read. As such, we use a specific type of SSD for this job. The
benefit the write cache provides is twofold:

  • Low latency response times for synchronous write
    requests (using write optimised SSDs)
  • Aggregation of small I/O into transaction groups
    resulting in fewer but larger (and therefore more efficient) physical I/O to
    the HDD.

The 2nd cache is the Adaptive Replacement Cache (ARC).
The ARC is held in DRAM which in a ZFS Storage Appliance is comparatively
pretty large (up to 3TB on a ZS4-4 controller pair). DRAM of course is the
fastest form of storage. It serves up I/O measured in nanoseconds compared to
the much slower microseconds of flash drives and milliseconds of hard drives.
We consider ZFS to be a DRAM first storage architecture and it is one key part
of the blinding performance ZFS can deliver.

The benefit of the ARC is threefold:

  • Provides the lowest latency reads possible (using large amounts of DRAM)
  • Stores a large amount of current hot data for
  • Reduces the read I/O demand on the hard drives

The 3rd cache is the Layer 2 Adaptive Replacement Cache (L2ARC) which uses large read optimised
SSDs (aka Readzillas) which have large capacity (up to 12.8TB per controller
pair) and great read performance, with more modest write and endurance
characteristics. The role of the L2ARC is to collect smaller pieces of data
that fall out of the ARC. This typically consists of data that is <32K in
size and has a reasonable level of re-use. The L2ARC is like a bench player, typically
not receiving much fanfare, but invaluable when called into action. The benefit
of the L2ARC is threefold:

  • Provides low latency reads (at flash storage
  • Stores a large amount of warm small I/O data
  • Protects the HDD from having to do (low productivity)
    small random IOPS

Using these three different technologies to act as three
different forms of cache has the effect that a final technology (being the
HDDs) is left to do what it does well. That is, storing large qualities of data
in an efficient manner using large physical I/O, therefore making the most of
every single rotation of the spindle.

Pulling this all together you not only get the fastest
storage, but also the most efficient, by simply using the right tool for the

Join the discussion

Comments ( 1 )
  • Deepa Monday, January 19, 2015

    Thank you for the info

Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.