Recipe for Sun Fire X4500 RAID-Z Config with Hot Spares

This configuration was discussed briefly in an earlier entry, but I wanted to dig a bit deeper on it by way of a better explanation.

The below command configures the disks in a Sun Fire X4500 as a single ZFS storage pool with 8 RAID-Z.4+1 stripes and 6 hot spares. ZFS dynamically stripes data in the pool across all of the stripes. Note that this runs over multiple lines.

zpool create -f  bigpool  \\
raidz c0t0d0 c1t0d0 c4t0d0 c6t0d0 c7t0d0 \\
raidz c1t1d0 c4t1d0 c5t1d0 c6t1d0 c7t1d0 \\
raidz c0t2d0 c4t2d0 c5t2d0 c6t2d0 c7t2d0 \\
raidz c0t3d0 c1t3d0 c5t3d0 c6t3d0 c7t3d0 \\
raidz c0t4d0 c1t4d0 c4t4d0 c6t4d0 c7t4d0 \\
raidz c0t5d0 c1t5d0 c4t5d0 c5t5d0 c7t5d0 \\
raidz c0t6d0 c1t6d0 c4t6d0 c5t6d0 c6t6d0 \\
raidz c0t7d0 c1t7d0 c4t7d0 c6t7d0 c7t7d0 \\
spare c0t1d0 c1t2d0 c4t3d0 c6t5d0 c7t6d0 c5t7d0

The server has six controllers each with eight disks and I have built the storage pool so that the hot spares are staggered across the controllers to spread I/O evenly and to enable me to build 8 RAID-Z stripes of equal length. Six hot spares may seem a lot, but it was either six spares or have fewer and make the RAID-Z stripes in the storage pool different lengths.

With 500 GB SATA-II disks this gives a usable capacity of 14 TB. Note that the capacity of a 500 GB disk as seen by Solaris is 466 GB.

root@x4500 # df -h bigpool  
Filesystem             size   used  avail capacity  Mounted on
bigpool                 14T    39K    14T     1%    /bigpool

root@x4500 # zpool status bigpool
  pool: bigpool
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        bigpool     ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c0t0d0  ONLINE       0     0     0
            c1t0d0  ONLINE       0     0     0
            c4t0d0  ONLINE       0     0     0
            c6t0d0  ONLINE       0     0     0
            c7t0d0  ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c1t1d0  ONLINE       0     0     0
            c4t1d0  ONLINE       0     0     0
            c5t1d0  ONLINE       0     0     0
            c6t1d0  ONLINE       0     0     0
            c7t1d0  ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c0t2d0  ONLINE       0     0     0
            c4t2d0  ONLINE       0     0     0
            c5t2d0  ONLINE       0     0     0
            c6t2d0  ONLINE       0     0     0
            c7t2d0  ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c0t3d0  ONLINE       0     0     0
            c1t3d0  ONLINE       0     0     0
            c5t3d0  ONLINE       0     0     0
            c6t3d0  ONLINE       0     0     0
            c7t3d0  ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c0t4d0  ONLINE       0     0     0
            c1t4d0  ONLINE       0     0     0
            c4t4d0  ONLINE       0     0     0
            c6t4d0  ONLINE       0     0     0
            c7t4d0  ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c0t5d0  ONLINE       0     0     0
            c1t5d0  ONLINE       0     0     0
            c4t5d0  ONLINE       0     0     0
            c5t5d0  ONLINE       0     0     0
            c7t5d0  ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c0t6d0  ONLINE       0     0     0
            c1t6d0  ONLINE       0     0     0
            c4t6d0  ONLINE       0     0     0
            c5t6d0  ONLINE       0     0     0
            c6t6d0  ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c0t7d0  ONLINE       0     0     0
            c1t7d0  ONLINE       0     0     0
            c4t7d0  ONLINE       0     0     0
            c6t7d0  ONLINE       0     0     0
            c7t7d0  ONLINE       0     0     0
        spares
          c0t1d0    AVAIL   
          c1t2d0    AVAIL   
          c4t3d0    AVAIL   
          c6t5d0    AVAIL   
          c7t6d0    AVAIL   
          c5t7d0    AVAIL   

errors: No known data errors

The performance of this configuration was discussed in a previous blog entry.

Comments:

Post a Comment:
Comments are closed for this entry.
About

Tim Thomas

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today