Wednesday Apr 18, 2012

Migrating Mirror Root Pool To New Laptop ... Recovery !!

A few years down the line and it's time to upgrade laptop as my once trusty laptop has decided to give up the ghost completely now, the on board graphics memory has decided it's had enough. So time to replace the motherboard or get a new one... so I got a new one.

So I think about how to migrate my nice Mirrored root pool configuration over to my new laptop, would be really nice to not have to actually install on the new laptop, and turns out I really don't have to and I've managed to keep the new laptop as a dual boot machine as well.

NOTE : Ensure the new disk/partition size you will be adding is at LEAST as large as the existing zpool device.

Here's what I did to achieve this :

1. Shrink Windows 7 Volume

Laptop comes pre-installed with Windows 7 and I'd like to actually keep a minimal install of this just in case I need IE, but I'd rather it not use the entire disk obviously. Boot into Windows 7, and run the disk management tool (diskmgmt.msc). In here you can remove Data volumes (assuming they don't have any actual data that you need). Then attempt to shrink the main C: volume down to a preferred size.

For me it's a 320GB disk so I'd like to shrink C: partition down to about 50GB. Right click on the partition in disk management tool and select shrink volume. Initially it would only allow me to shrink to 100GB, as windows was encountering some unmovable files. To resolve, I turned of Paging, Hibernation and Restore and Backup, I could then shrink the volume. Before rebooting out of windows ensure to re-enable these services.

Once shrunk I have about 260GB of free unused space to use, to ensure windows functions later on, I create one large NTFS partition on this unused space, we'll be redefining this later on for our root zpool.

2. Configure New Disk For Solaris.

Before attaching our USB mirror and the device we need to perform a few configuration steps on the new Laptop's internal disk. Specifically creating a Solaris2 partition and labeling it.

Boot new laptop into a Solaris Live CD or Text Install environment, and get to a shell prompt. From here (as root), run format(1) to create a Solaris2 partition and label the partition.

  $ format
  Searching for disks...done

  AVAILABLE DISK SELECTIONS:
         0. c5t0d0 
            /pci@0,0/pci1043,15d7@1f,2/disk@0,0

From this we can see the internal disk is device c5t0d0, using the format(1) command and fdisk option we need to remove the NTFS partition we originally created and replace it with a Solaris2 partion.

  $ format c5t0d0

From menu displayed, choose fdisk option :

  $ format> fdisk

This should show what current partition layout is. For me it's showing three NTFS partitions, the largest being partition 3, which is the one I created in Windows 7 of all the available left over space. Simply delete this partition via option 3 from the displayed menu. Before deleting make a mental note of the size of the partition specifically the starting cylinder and the length in cylinders. If using the whole disk then you can simply specify 100%.

Now to create the new Solaris2 partition choose Option 1 from the displayed menu :

  SELECT ONE OF THE FOLLOWING:
     1. Create a partition
     2. Specify the active partition
     3. Delete a partition
     4. Change between Solaris and Solaris2 Partition IDs
     5. Edit/View extended partitions
     6. Exit (update disk configuration and exit)
     7. Cancel (exit without updating disk configuration)
  Enter Selection: 1

  Select the partition type to create:
   1=SOLARIS2  2=UNIX        3=PCIXOS     4=Other
   5=DOS12     6=DOS16       7=DOSEXT     8=DOSBIG
   9=DOS16LBA  A=x86 Boot    B=Diagnostic C=FAT32
   D=FAT32LBA  E=DOSEXTLBA   F=EFI        0=Exit? 1

  Specify the percentage of disk to use for this partition
  (or type "c" to specify the size in cylinders). c
 
  Enter starting cylinder number: 7769
  Enter partition size in cylinders : 41144

  Should this become the active partition? If yes, it  will be activated
  each time the computer is reset or turned on.
  Please type "y" or "n". y

As can be seen from above, choose option 1 to create a new partition, select 1, for SOLARIS2 type partition, and 100% if using the entire disk, other wise specify the starting cylinder and number of cylinders to use. These you will have written down from the NTFS partition you deleted. You may need to +1 to the starting point and -1 from total number of cylinders, and finally "y" to make partition active. You should then see something like the following :

               Total disk size is 38912 cylinders
               Cylinder size is 16065 (512 byte) blocks

                                                 Cylinders
        Partition   Status    Type          Start   End   Length    %
        =========   ======    ============  =====   ===   ======   ===
            1                 NTFS              1     10       10    1
            1                 NTFS             10   7768      700   18  
            1       Active    Solaris2       7768  38912    31145   80


  SELECT ONE OF THE FOLLOWING:
     1. Create a partition
     2. Specify the active partition
     3. Delete a partition
     4. Change between Solaris and Solaris2 Partition IDs
     5. Edit/View extended partitions
     6. Exit (update disk configuration and exit)
     7. Cancel (exit without updating disk configuration)
  Enter Selection: 

Choose 6 to update disk configuration and exist fdisk and back to format(1) menu.

Now to label the disk, ensure a VTOC label is on the Solaris partition, sizes listed above and below are examples only and may not be what you will see on your actual disks.

 
  $ format> partition

  PARTITION MENU:
        0      - change '0' partition
        1      - change '1' partition
        2      - change '2' partition
        3      - change '3' partition
        4      - change '4' partition
        5      - change '5' partition
        6      - change '6' partition
        7      - change '7' partition
        select - select a predefined table
        modify - modify a predefined partition table
        name   - name the current table
        print  - display the current table
        label  - write partition map and label to the disk
        ! - execute , then return
        quit
  $ partition> print
  Current partition table (original):
  Total disk cylinders available: 9419 + 2 (reserved cylinders)

  Part     Tag    Flag   Cylinders      Size         Blocks
  0 unassigned     wm     0             0       (0/0/0)             0
  1 unassigned     wm     0             0       (0/0/0)             0
  2     backup     wm     0 - 19453   149.03GB  (19454/0/0) 312528510 
  3 unassigned     wm     0             0       (0/0/0)             0
  4 unassigned     wm     0             0       (0/0/0)             0
  5 unassigned     wm     0             0       (0/0/0)             0
  6 unassigned     wm     0             0       (0/0/0)             0
  7 unassigned     wm     0             0       (0/0/0)             0
  8       boot     wm     0 -     0     7.84MB  (1/0/0)         16065
  9 unassigned     wm     0             0       (0/0/0)             0

  $ partition>

We need to ensure slice 0 is configured, in my case I configure it to be the whole size of the available partition, then actually label the partition.


  $ partition> 0
  Part     Tag    Flag   Cylinders      Size         Blocks
  0 unassigned     wm     0             0       (0/0/0)             0

  Enter partition id tag[unassigned]: root
  Enter partition permission flags[wm]: wm
  Enter new starting cyl[1]: 1
  Enter partition size[0b, 0c, 0e, 0mb, 0gb]: 19453c
  $ partition> print
  Current partition table (original):
  Total disk cylinders available: 9419 + 2 (reserved cylinders)

  Part     Tag    Flag   Cylinders      Size         Blocks
  0       root     wm     1 - 19453   149.02GB  (19453/0/0) 312512445
  1 unassigned     wm     0             0       (0/0/0)             0
  2     backup     wm     0 - 19453   149.03GB  (19454/0/0) 312528510 
  3 unassigned     wm     0             0       (0/0/0)             0
  4 unassigned     wm     0             0       (0/0/0)             0
  5 unassigned     wm     0             0       (0/0/0)             0
  6 unassigned     wm     0             0       (0/0/0)             0
  7 unassigned     wm     0             0       (0/0/0)             0
  8       boot     wm     0 -     0     7.84MB  (1/0/0)         16065
  9 unassigned     wm     0             0       (0/0/0)             0

  $ partition> label
  Ready to label disk, continue? y
  $ partition> quit

Again ensure disk is definitely labeled from the format prompt :

  $ format> label
  Ready to label disk, continue? y

Choose label from the format(1) menu, and select "y". If prompted for specific label type, ensure to choose SMI. Once labeled choose quit to get back to shell prompt.

  $ format> quit

3. Replacing Mirror Device.

My old laptop configuration consisted of an internal 200GB disk mirrored with an external 300GB USB drive, this resulted in a 200GB mirrored root pool. To migrate to new laptop I simply need my USB disk.

Again ensure new laptop is booted into a Solaris Live DVD or Text Install environment, and get to a prompt. Ensure the USB drive is attached to the machine, then simply import the degraded rpool.

  $ zpool import -f rpool
  $ zpool status
    pool: rpool
   state: DEGRADED
  status: One or more devices could not be opened. Sufficient replicas exist for
          the pool to continue functioning in a degraded state.
  action: Attach the missing device and online it using 'zpool online'
     see: http://www.sun.com/msg/ZFS-8000-2Q
    scan: resilvered 181M in 0h0m with 0 errors on Fri Mar 30 10:32:13 2012
  config:

	NAME                     STATE     READ WRITE CKSUM
	rpool                    DEGRADED     0     0     0
	  mirror-0               DEGRADED     0     0     0
	    2000330655777608537  UNAVAIL      0     0     0 was /dev/dsk/c4t0d0s0
	    c6t0d0s0             ONLINE       0     0     0

  errors: No known data errors

The UNAVAIL device from above is the original laptop's internal drive, which obviously is not available on the new machine. c6t0d0s0 is the USB device.

Now all we have to do is replace the unavailable device with the new internal disk partition we created above. Make sure you specify slice 0 e.g. s0

  $ zpool replace -f rpool 2000330655777608537 c5t0d0s0

The old device will now get replaced with the new internal disk device and depending on the size will may take a few hours to resilver everything from the USB disk onto the new internal device. Once completed zpool status should look like following :

  $ zpool status
    pool: rpool
   state: ONLINE
  config:

	NAME          STATE     READ WRITE CKSUM
	rpool         ONLINE       0     0     0
	  mirror-0    ONLINE       0     0     0
	    c5t0d0s0  ONLINE       0     0     0
	    c6t0d0s0  ONLINE       0     0     0

  errors: No known data errors

4. Install Grub To Internal Disk

Finally we need to install grub to the internal disk and ensure grub menu entries are correct for booting.

  $ installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c5t0d0s0

If you have other Dual bootable OS's you will need to edit /rpool/boot/grub/menu.lst and add entries for them here otherwise you won't be able to get to them. In my previous configuration I had consumed the entire disk on both devices for my root pool, in this case the grub menu entries are exactly the same for locating the disk and partition e.g. findroot (0,a) However my new scenario is slightly different, I am consuming the whole disk on the external USB disk but only partition 3 on the internal disk. Because of this I will need two separate grub menu entries depending on which disk I'd like to boot from. Here's an example of what mine looks like :

title Windows 7
rootnoverify (hd0,1)
chainloader +1

title Internal Disk Solaris 11
findroot (pool_rpool, 2, a)
bootfs rpool/ROOT/s11
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -v
module$ /platform/i86pc/$ISADIR/boot_archive

title USB Disk Solaris 11
findroot (pool_rpool, 0, a)
bootfs rpool/ROOT/s11
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -v
module$ /platform/i86pc/$ISADIR/boot_archive

Notice from above the only difference between the Internal disk and the USB is the findroot line for the internal disk showing 2 (partition 3), and on the USB disk 0 (partition 1).

Now export your new rpool and reboot, ensuring to remove the Live DVD/Text Install disc so that you boot from your new rpool.

  $ zpool export rpool
  $ init 5

Power on your laptop and at the grub prompt edit one of the Solaris entries and put "-r" at end of kernel line, and then select 'b' to boot. This will boot solaris in reconfigure mode and will attempt to ensure all the correct devices are loaded for your new laptop. This only needs to be done once e.g. initial boot.

Now you may find other issues with various devices on your laptop, like display, wifi etc... but that's an entirely new blog...

5. Auto expanding to maximize available space.

In my situation my rpool size was constrained by my original Laptop's internal disk size of 200GB. After following the above steps I now have a 200GB rpool recreated however I know the USB drive is 300GB in size and I know the SOLARIS2 partition on my new internal disk is 260GB, so I have 60GB of unused space.

Zpools have a property 'autoexpand' which by default is set to 'off'. This property if set to true will auto expand the pool to whatever space it can on the devices attached to the pool. So I can now simply set this property to 'on' and I will get the unused space automatically in my pool: Before :

  $ zpool list
  NAME   SIZE  ALLOC   FREE  CAP  DEDUP  HEALTH  ALTROOT
  rpool  180G   135G  51.2G  72%  1.00x  ONLINE  -
Reset the autoexpand property, and re run zpool list.
  $ zpool set autoexpand=on rpool
  $ zpool list
  NAME   SIZE  ALLOC   FREE  CAP  DEDUP  HEALTH  ALTROOT
  rpool  260G   135G 131.2G  51%  1.00x  ONLINE  -

If the size does not automatically expand as you expect you may need need to check the s0 size on your disks, one of them may be hard set to the lower size than the actual space on that solaris partition. Changing the VTOC s0/s2 (change both if necessary), will not effect anything as long as you are not changing the starting cylinder. After the expansion has completed, reset autoexpand property back to it's default of off.

  $ zpool set autoexpand=off rpool

6. Ensuring boot succeeds when one device is missing.

Potentially a bug but on a default solaris install when I disconnect the USB device the laptop fails to boot, mainly as the root pool is in a degraded state because a device is UNAVAIL. Pools now have a property "failmode". By default this is set to 'wait'. If you change this to a value of "continue', boot should succeed.
  $ zpool set failmode=continue rpool

Friday May 07, 2010

Bootable USB Mirror

Recently my laptop hard drive simply decided it had enough, unfortunately I had become a little lazy when it came to doing backup's so I got bitten pretty hard.

To ensure I don't get hit by this again, I got myself a USB portable drive and decided to use existing ZFS technology available on Solaris. I could have just attached the new device, set up a new pool on that device and performed regular zfs send's and I would have had a nice backup solution.

Another means would be to attach the new USB drive to my existing single pool laptop drive as a mirror and my data would get backed up automatically every time I attached the USB drive.

But I thought I'd go one step further, why not set up the attached device as bootable, and that way I can simply attach it to any other laptop/PC and boot up via USB and have the exact same environment that I have on my original laptop.

Here's the steps I performed to do this :

1. Format/Partition/Label New USB Disk

New drive needs to be specifically partitioned for Solaris. We need to identify the physical devices. Firstly lets identify the current laptop hard drive device and pool setup.

  $ pfexec zpool status
    pool: rpool
   state: ONLINE
  config:

	NAME          STATE     READ WRITE CKSUM
	rpool         ONLINE       0     0     0
	    c5t0d0s0  ONLINE       0     0     0

  errors: No known data errors

From above we can see laptop disk device is c5t0d0s0, which is a 200GB drive with a single default pool "rpool" for the entire disk.

Using the format(1) command we can identify the USB drive, ensure it's attached and power is on.

  $ pfexec format
  Searching for disks...done


  AVAILABLE DISK SELECTIONS:
         0. c5t0d0 
            /pci@0,0/pci1043,15d7@1f,2/disk@0,0
         1. c6t0d0 
            /pci@0,0/pci1043,15d7@1d,7/storage@1/disk@0,0

From this we can see the USB disk is device c6t0d0, using the format(1) command and fdisk option we need to remove any existing partitions and create just one new partition of type SOLARIS2 for 100% of the disk.

  $ pfexec format c6t0d0

From menu displayed, choose fdisk option :

  $ format> fdisk

This should show what current partition layout is, if there are existing partitions just delete them via option 3 from the displayed menu.

Once all existing partitions have been deleted, choose option 1 from displayed menu :

  SELECT ONE OF THE FOLLOWING:
     1. Create a partition
     2. Specify the active partition
     3. Delete a partition
     4. Change between Solaris and Solaris2 Partition IDs
     5. Edit/View extended partitions
     6. Exit (update disk configuration and exit)
     7. Cancel (exit without updating disk configuration)
  Enter Selection: 1

  Select the partition type to create:
   1=SOLARIS2  2=UNIX        3=PCIXOS     4=Other
   5=DOS12     6=DOS16       7=DOSEXT     8=DOSBIG
   9=DOS16LBA  A=x86 Boot    B=Diagnostic C=FAT32
   D=FAT32LBA  E=DOSEXTLBA   F=EFI        0=Exit? 1

  Specify the percentage of disk to use for this partition
  (or type "c" to specify the size in cylinders). 100

  Should this become the active partition? If yes, it  will be activated
  each time the computer is reset or turned on.
  Please type "y" or "n". y

As can be seen from above, choose option 1 to create a new partition, select 1, for SOLARIS2 type partition, and 100% for entire disk and finally "y" to make partition active. You should then see something like the following :

               Total disk size is 38913 cylinders
               Cylinder size is 16065 (512 byte) blocks

                                                 Cylinders
        Partition   Status    Type          Start   End   Length    %
        =========   ======    ============  =====   ===   ======   ===
            1       Active    Solaris2          1  38912    38912    100


  SELECT ONE OF THE FOLLOWING:
     1. Create a partition
     2. Specify the active partition
     3. Delete a partition
     4. Change between Solaris and Solaris2 Partition IDs
     5. Edit/View extended partitions
     6. Exit (update disk configuration and exit)
     7. Cancel (exit without updating disk configuration)
  Enter Selection: 

Choose 6 to update disk configuration and exist fdisk and back to format(1) menu.

Now to label the disk, ensure a VTOC label is on the Solaris partition, sizes listed below are examples only and may not be what you will see on your actual disks.

 format> partition

PARTITION MENU:
        0      - change '0' partition
        1      - change '1' partition
        2      - change '2' partition
        3      - change '3' partition
        4      - change '4' partition
        5      - change '5' partition
        6      - change '6' partition
        7      - change '7' partition
        select - select a predefined table
        modify - modify a predefined partition table
        name   - name the current table
        print  - display the current table
        label  - write partition map and label to the disk
        ! - execute , then return
        quit
partition> print
Current partition table (original):
Total disk cylinders available: 9419 + 2 (reserved cylinders)

Part     Tag    Flag   Cylinders      Size         Blocks
0 unassigned     wm     0             0       (0/0/0)             0
1 unassigned     wm     0             0       (0/0/0)             0
2     backup     wm     0 - 19453   149.03GB  (19454/0/0) 312528510 
3 unassigned     wm     0             0       (0/0/0)             0
4 unassigned     wm     0             0       (0/0/0)             0
5 unassigned     wm     0             0       (0/0/0)             0
6 unassigned     wm     0             0       (0/0/0)             0
7 unassigned     wm     0             0       (0/0/0)             0
8       boot     wm     0 -     0     7.84MB  (1/0/0)         16065
9 unassigned     wm     0             0       (0/0/0)             0

partition>

We need to ensure slice 0 is configured, in my case I configure it to be the whole size of the disk, also ensure disk is labeled :


  partition> 0
  Part     Tag    Flag   Cylinders      Size         Blocks
  0 unassigned     wm     0             0       (0/0/0)             0

  Enter partition id tag[unassigned]: root
  Enter partition permission flags[wm]: wm
  Enter new starting cyl[1]: 1
  Enter partition size[0b, 0c, 0e, 0mb, 0gb]: 19453c
  partition> print
  Current partition table (original):
  Total disk cylinders available: 9419 + 2 (reserved cylinders)

  Part     Tag    Flag   Cylinders      Size         Blocks
  0       root     wm     1 - 19453   149.02GB  (19453/0/0) 312512445
  1 unassigned     wm     0             0       (0/0/0)             0
  2     backup     wm     0 - 19453   149.03GB  (19454/0/0) 312528510 
  3 unassigned     wm     0             0       (0/0/0)             0
  4 unassigned     wm     0             0       (0/0/0)             0
  5 unassigned     wm     0             0       (0/0/0)             0
  6 unassigned     wm     0             0       (0/0/0)             0
  7 unassigned     wm     0             0       (0/0/0)             0
  8       boot     wm     0 -     0     7.84MB  (1/0/0)         16065
  9 unassigned     wm     0             0       (0/0/0)             0

  partition> label
  Ready to label disk, continue? y
  partition> quit

Again ensure disk is definitely labeled from the format prompt :

  $ format> label
  Ready to label disk, continue? y

Choose label from the format(1) menu, and select "y". If prompted for specific label type, ensure to choose SMI. Once labeled choose quit to get back to shell prompt.

  $ format> quit

3. Attach USB Disk To Main Pool Creating Mirror

Now to attach the new USB drive to the existing rpool and thus creating a mirror. This time ensure you use s0 (slice 0) at end of device.

  $ pfexec zpool attach -f rpool c5t0d0s0 c6t0d0s0

Once this completes run zpool status again, and you will see the new device as part of your new mirror configuration, and will also show percentage re-silvering. It took 1hr 45mins to resilver a 200GB disk, and 120GB of data.

  $ zpool status
    pool: rpool
   state: ONLINE
  config:

	NAME          STATE     READ WRITE CKSUM
	rpool         ONLINE       0     0     0
	  mirror-0    ONLINE       0     0     0
	    c5t0d0s0  ONLINE       0     0     0
	    c6t0d0s0  ONLINE       0     0     0  1.18G resilvered

  errors: No known data errors

4. Install Grub on USB Disk

Finally we need to install grub on the USB disk to ensure it is bootable :

  $ pfexec installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c6d0s0

5. Some Caveats

Whilst I definitely think this is a great backup solution, it doesn't come without some caveats/kinks that I've come across over the first few days with this setup.

  • Ensure your USB disk is at least as big as your laptop drive. In a ZFS mirror configuration, the smallest disk dictates the size of the pool.
  • Always have your USB drive connected a boot time. For some reason Solaris will not boot unless your drive is connected at boot time. This to me seems like a bug.
  • When shutting down, remember the exact USB port your USB drive was connected to, on some occasions I've noticed Solaris would not boot unless the USB drive was connected to the exact same port when shutdown.
  • NEVER shutdown whilst your USB drive is on offline mode. You can take your USB drive offline via zfs offline pool disk, however if you shutdown whilst it is offline (and disconnected) Solaris simply hangs when attempting to boot, again another possible bug. To recover from this situation I had to do the following :
    • Boot into LiveCD environment of a Solaris build of the same or more recent than currently installed build on your laptop. Ensure USB drive is connected when booting.
    • Once booted import your pool, and bring USB drive back online :
        $ zpool import -f rpool
        $ zpool online rpool %ltusbdisk%gt
      
    • Now reboot and you should be good to go.
About

Install engineer at Oracle who is passionate about Music, Sport and has a soft spot for Solaris

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today