Talking to customers this week about the Oracle Solaris ZFS device removal feature brought up a few key points that I would like to emphasize as a follow-on to this previous blog:
Customers who have leveraged this flexibility for years now are asking what exactly is the difference between detaching a device or removing device? This difference is best illustrated with a mirrored storage pool. The key difference is that detaching a device from a mirrored pool does not reduce the pool capacity. Removing a mirrored vdev in the Solaris 11.4 release does reduce the pool capacity. See the example below.
Also illustrating my point that ZFS always had device flexibility was a comment from a customer a few years ago, who said about ZFS device removal, "Forget it...I don't need it any more!"
I have 2 mirrored pools, tank and tank-clone, roughly the equivalent amount of data and pool capacity:
# zpool list tank tank-clone NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 928G 56.6G 871G 6% 2.00x ONLINE - tank-clone 928G 56.3G 872G 6% 1.00x ONLINE - # zpool status tank tank-clone pool: tank state: ONLINE scan: resilvered 19.8M in 12s with 0 errors on Mon Apr 16 14:41:28 2018 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c4t2d0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 c3t4d0 ONLINE 0 0 0 errors: No known data errors pool: tank-clone state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank-clone ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t0d0 ONLINE 0 0 0 c6t0d0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c5t0d0 ONLINE 0 0 0 c2t0d0 ONLINE 0 0 0 errors: No known data errors
Detach devices from tank. Removing pool redundancy is not recommended but to illustrate a point that detaching a device from a mirrored configuration doesn't change the pool's overall capacity:
# zpool detach tank c4t2d0 # zpool detach tank c3t4d0 # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 928G 56.6G 871G 6% 2.00x ONLINE - # zpool status tank pool: tank state: ONLINE scan: resilvered 19.8M in 12s with 0 errors on Mon Apr 16 14:41:28 2018 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 errors: No known data errors
Next, remove a mirrored vdev from tank-clone and review the pool capacity change:
# zpool remove tank-clone mirror-1 # zpool status tank-clone pool: tank-clone state: ONLINE scan: resilvered 28.1G in 13m17s with 0 errors on Wed Apr 25 09:36:35 2018 config: NAME STATE READ WRITE CKSUM tank-clone ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t0d0 ONLINE 0 0 0 c6t0d0 ONLINE 0 0 0 errors: No known data errors # zpool list tank tank-clone NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 928G 56.6G 871G 6% 2.00x ONLINE - tank-clone 464G 56.7G 407G 12% 1.00x ONLINE -
In summary, Solaris 11.4 device removal brings additional flexibility for managing your data and devices with ZFS.