Thursday Feb 23, 2012

Resizing a mirror with ZFS - a VDI ZFS pool!

My everyday VDI setup is a single host Solaris 10 server with four disks, two of which are mirrored for VDI use. This ZFS pool, the main storage of my sole deskto provider, which I called "VDI" in a spurt of creativity, started to seriously run out of space when I tried to create a pool of Windows 7 desktops.

It was time to out to use a stash of 500GB disks that were left over from another system that no longer needed them to replace the existing 250GB ones, in other words, double my capacity. After all, 11 Windows XP desktops, 5 Ubuntu desktops and 3 Windows 7 desktops is not a bad challenge for 250GB, but I needed more! Someone, somewhere told me this could not be done. I had to try it because I had nothing to lose.

The following snapshot of commands took place after replacing the first disk in the mirror,
c0t3d0 , which I omitted since it wasn't particularly interesting, after all, replacing a disk in a ZFS mirror is straight forward and adds no capacity to the mirror until the second disk is replaced.

- Find the disk you're after...
vdidc# zpool status VDI
  pool: VDI
 state: ONLINE
 scrub: resilver completed after 14h51m with 0 errors on Thu Feb 23 01:58:52 2012
config:

        NAME        STATE     READ WRITE CKSUM
        VDI         ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c0t4d0  ONLINE       0     0     0
            c0t3d0  ONLINE       0     0     0  185G resilvered

- It looks like it's c0t4d0. Let's make sure.
vdidc# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t1d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
          /pci@0,0/pci108e,5354@1f,2/disk@1,0
       1. c0t2d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
          /pci@0,0/pci108e,5354@1f,2/disk@2,0
       2. c0t3d0 <ATA-HITACHI HDS7250S-AJ0A-465.76GB>
          /pci@0,0/pci108e,5354@1f,2/disk@3,0
       3. c0t4d0 <ATA-SEAGATE ST32502N-SU0B-232.88GB>  <--250GB disk.
          /pci@0,0/pci108e,5354@1f,2/disk@4,0

- Now we break it from the mirror.
vdidc# zpool detach VDI c0t4d0
vdidc# zpool status VDI
  pool: VDI
 state: ONLINE
 scrub: resilver completed after 14h51m with 0 errors on Thu Feb 23 01:58:52 2012
config:

        NAME        STATE     READ WRITE CKSUM
        VDI         ONLINE       0     0     0
        c0t3d0    ONLINE       0     0     0  185G resilvered

- Physical work: replace the drive itself. No youtube video for this simple task, but unfortunately, Solaris 10 refuses to see the disk automatically.

vdidc# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t1d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
          /pci@0,0/pci108e,5354@1f,2/disk@1,0
       1. c0t2d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
          /pci@0,0/pci108e,5354@1f,2/disk@2,0
       2. c0t3d0 <ATA-HITACHI HDS7250S-AJ0A-465.76GB>
          /pci@0,0/pci108e,5354@1f,2/disk@3,0

- Let's add the disk manually
vdidc# cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c0t0d0   cd/dvd      connected    configured   ok
sata0/1::dsk/c0t1d0   disk         connected    configured    ok
sata0/2::dsk/c0t2d0   disk         connected    configured    ok
sata0/3::dsk/c0t3d0   disk         connected    configured    ok
sata0/4                        disk         connected    unconfigured unknown <-- THIS IS IT
...

vdidc# cfgadm -c configure sata0/4

vdidc# cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c0t0d0   cd/dvd     connected    configured   ok
sata0/1::dsk/c0t1d0   disk         connected    configured   ok
sata0/2::dsk/c0t2d0   disk         connected    configured   ok
sata0/3::dsk/c0t3d0   disk         connected    configured   ok
sata0/4::dsk/c0t4d0   disk         connected    configured   ok <-- DONE
...

vdidc# format
Searching for disks...done

c0t4d0: configured with capacity of 465.74GB

AVAILABLE DISK SELECTIONS:
       0. c0t1d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
          /pci@0,0/pci108e,5354@1f,2/disk@1,0
       1. c0t2d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
          /pci@0,0/pci108e,5354@1f,2/disk@2,0
       2. c0t3d0 <ATA-HITACHI HDS7250S-AJ0A-465.76GB>
          /pci@0,0/pci108e,5354@1f,2/disk@3,0
       3. c0t4d0 <ATA-HITACHIHDS7250S-AJ0A cyl 60799 alt 2 hd 255 sec 63>  <-- CONFIRMED
          /pci@0,0/pci108e,5354@1f,2/disk@4,0

- Now that the disk is in the fold it's time to attach it to the pool and get our mirror back!
vdidc# zpool attach VDI c0t3d0 c0t4d0
vdidc# zpool status VDI
  pool: VDI
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
 scrub: resilver in progress for 0h0m, 0.02% done, 54h52m to go
config:

        NAME        STATE     READ WRITE CKSUM
        VDI         ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c0t3d0  ONLINE       0     0     0
            c0t4d0  ONLINE       0     0     0  38.8M resilvered

errors: No known data errors

- A few hours later, the mirror has been upgraded and resilvered, but is the capacity there yet? No!

vdidc# zpool list
NAME    SIZE  ALLOC   FREE    CAP  HEALTH  ALTROOT
VDI         232G   188G  43.7G    81%  ONLINE  -
rpool       464G   305G   159G    65%  ONLINE  -

- To fix this was just too easy...
vdidc# zpool set autoexpand=on VDI
vdidc# zpool list
NAME    SIZE  ALLOC   FREE    CAP  HEALTH  ALTROOT
VDI         464G   188G   276G    40%  ONLINE  -
rpool       464G   305G   159G    65%  ONLINE  -

That's it, mirrored replaced and upgraded, disk capacity expanded, downtime required: ZERO, no desktops taken offline. But the disks did take a proper hammering during the resilvering stages! Gotta love ZFS.

About

A thin thinker down under

Search

Categories
Archives
« February 2012 »
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
24
25
26
27
28
29
   
       
Today