X

The ZFS blog provides product announcements and technological insights

  • July 24, 2015

Reducing Storage Costs with ZFS Compression

Cindy Swearingen
Principal Product Manager

This blog describes how to use ZFS compression as a simple and flexible way to reduce the power, cooling, and floor space costs that are associated with storing data.

Oracle Solaris 11.3 includes lz4 compression. You can display supported compression algorithms on your system like this: 

# zfs help compression
compression (property)
Editable, Inheritable
Accepted values: on | off | lzjb | gzip | gzip-[1-9] | zle | lz4

When ZFS compression is enabled on a file system, all newly written data is compressed. If you are storing new, compressible data, the syntax is simple:

# zfs create -o compression=on tank/data

ZFS file system properties are flexible because you can set them on an individual file system or an entire pool. For example:

# zfs set compression=on pond

The above syntax means that all newly written data to this pool is compressed.

If you have compressible data but are concerned about the impact on system resources, consider trying lz4 compression. For example:

# zfs create -o compression=lz4 tank/cdata
cannot create 'tank/cdata': pool must be upgraded to set this property
or value

This error means that the pool version needs to be updated to pool version 37 to support lz4 compression.

# zpool upgrade tank
This system is currently running ZFS pool version 37.

Successfully upgraded 'tank' from version 35 to version 37

# zfs create -o compression=lz4 tank/cdata

 *************************

 More clarification added around root pool behavior:

Currently, neither gzip nor lz4 compression is supported on root pools. If you attempt to set either gzip or lz4 compression on the current BE file system, you see an error similar to the following:

# zfs set compression=lz4 rpool/ROOT/s11u3

cannot set property for 'rpool/ROOT/s11u3': property setting is not
allowed on bootable datasets 

If you set either gzip or lz4 compression on a root pool, no error occurs. (I filed a bug about this). If you attempt to pkg update the current BE, the operation eventually fails. You will see a message similar to this:

BootmgmtError: operation not supported on this type of pool

Use the following steps to recover:

 1. Set default compression (lzjb) on the root pool:

# zfs set compression=on rpool

2. Remove the BE that was created during the (failed) pkg update process.

# beadm destroy s11u3-1 

************************* 

In summary, ZFS compression is simple, flexible, and can reduce your storage footprint.

Solaris customers are reporting compression ratios in the 2X - 15X range, depending on their workloads, which can translate into huge cost savings. Give it a try.

Join the discussion

Comments ( 7 )
  • guest Saturday, March 5, 2016

    compare it with lz4 in openzfs, it seems to be much more effective


  • Cindys Monday, March 7, 2016

    Our LZ4 compression testing did not match the illumos LZ4 compression testing. One of customers, who was an early ZFS adopter, also could not match the illumos LZ4 compression testing so if you have data that matches the illumos LZ4 compression, I would like see it.

    Thanks, Cindy


  • guest Thursday, August 18, 2016

    Hi Cindy,

    looking around some cpu increase on a solaris 11.3 upgrade with no compression or dedup active on the pool or zfs fs found this trace call

    24 80353 zfs_lzjb_compress:return 490

    24 80353 zfs_lzjb_compress:return 490

    24 80353 zfs_lzjb_compress:return 490

    Do you know how is this new zfs_lzjb_compress used ? it seems active by default on all the pools some how


  • guest Thursday, August 18, 2016

    Hi Cindy,

    looking around some cpu increase on a solaris 11.3 upgrade with no compression or dedup active on the pool or zfs fs found this trace call

    24 80353 zfs_lzjb_compress:return 490

    24 80353 zfs_lzjb_compress:return 490

    24 80353 zfs_lzjb_compress:return 490

    Do you know how is this new zfs_lzjb_compress used ? it seems active by default on all the pools some how


  • Cindys Thursday, August 18, 2016

    Hi Guest,

    We looked up the zfs_lzjb_compress function and we believe this function is compressing metadata only. We don't believe this is the cause of the extra CPU cycles. Have you collected FMA data and ruled out any ongoing issues?

    Thanks, Cindy


  • Cindys Thursday, September 22, 2016

    I think I accidentally removed this comment (comment management is kind of clunky) so I include it here:

    guest wrote:

    I'm running 11.3 with SRU 11.3.9.4.0.

    It let me enable lz4 compression on the root BE with out any complaint. But did fail with "pkg: An unexpected error happened during update: operation not supported on this type of pool" when I issued a pkg update to SRU 11.3.11.6.0.

    Any bugid or RFE to follow for when a higher compression might be allowed on the root BE?


  • Cindys Thursday, September 22, 2016

    Yes, I understand this scenario and bugs are filed to provide an error message when you try to apply the unsupported compression algorithms (lz4 and gzip) on the root pool or the root dataset in a upcoming Solaris release.

    The primary bug to track for support of the LZ4 on a root pool is this one:

    https://bug.oraclecorp.com/pls/bug/webbug_print.show?c_rptno=20736930

    Thanks, Cindy


Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.