Bootable USB Mirror
By User12821160-Oracle on May 07, 2010
To ensure I don't get hit by this again, I got myself a USB portable drive and decided to use existing ZFS technology available on Solaris. I could have just attached the new device, set up a new pool on that device and performed regular zfs send's and I would have had a nice backup solution.
Another means would be to attach the new USB drive to my existing single pool laptop drive as a mirror and my data would get backed up automatically every time I attached the USB drive.
But I thought I'd go one step further, why not set up the attached device as bootable, and that way I can simply attach it to any other laptop/PC and boot up via USB and have the exact same environment that I have on my original laptop.
Here's the steps I performed to do this :
1. Format/Partition/Label New USB Disk
New drive needs to be specifically partitioned for Solaris. We need to identify the physical devices. Firstly lets identify the current laptop hard drive device and pool setup.
$ pfexec zpool status pool: rpool state: ONLINE config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c5t0d0s0 ONLINE 0 0 0 errors: No known data errors
From above we can see laptop disk device is c5t0d0s0, which is a 200GB drive with a single default pool "rpool" for the entire disk.
Using the format(1) command we can identify the USB drive, ensure it's attached and power is on.
$ pfexec format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c5t0d0
/pci@0,0/pci1043,15d7@1f,2/disk@0,0 1. c6t0d0 /pci@0,0/pci1043,15d7@1d,7/storage@1/disk@0,0
From this we can see the USB disk is device c6t0d0, using the format(1) command and fdisk option we need to remove any existing partitions and create just one new partition of type SOLARIS2 for 100% of the disk.
$ pfexec format c6t0d0
From menu displayed, choose fdisk option :
$ format> fdisk
This should show what current partition layout is, if there are existing partitions just delete them via option 3 from the displayed menu.
Once all existing partitions have been deleted, choose option 1 from displayed menu :
SELECT ONE OF THE FOLLOWING: 1. Create a partition 2. Specify the active partition 3. Delete a partition 4. Change between Solaris and Solaris2 Partition IDs 5. Edit/View extended partitions 6. Exit (update disk configuration and exit) 7. Cancel (exit without updating disk configuration) Enter Selection: 1 Select the partition type to create: 1=SOLARIS2 2=UNIX 3=PCIXOS 4=Other 5=DOS12 6=DOS16 7=DOSEXT 8=DOSBIG 9=DOS16LBA A=x86 Boot B=Diagnostic C=FAT32 D=FAT32LBA E=DOSEXTLBA F=EFI 0=Exit? 1 Specify the percentage of disk to use for this partition (or type "c" to specify the size in cylinders). 100 Should this become the active partition? If yes, it will be activated each time the computer is reset or turned on. Please type "y" or "n". y
As can be seen from above, choose option 1 to create a new partition, select 1, for SOLARIS2 type partition, and 100% for entire disk and finally "y" to make partition active. You should then see something like the following :
Total disk size is 38913 cylinders Cylinder size is 16065 (512 byte) blocks Cylinders Partition Status Type Start End Length % ========= ====== ============ ===== === ====== === 1 Active Solaris2 1 38912 38912 100 SELECT ONE OF THE FOLLOWING: 1. Create a partition 2. Specify the active partition 3. Delete a partition 4. Change between Solaris and Solaris2 Partition IDs 5. Edit/View extended partitions 6. Exit (update disk configuration and exit) 7. Cancel (exit without updating disk configuration) Enter Selection:
Choose 6 to update disk configuration and exist fdisk and back to format(1) menu.
Now to label the disk, ensure a VTOC label is on the Solaris partition, sizes listed below are examples only and may not be what you will see on your actual disks.
format> partition PARTITION MENU: 0 - change '0' partition 1 - change '1' partition 2 - change '2' partition 3 - change '3' partition 4 - change '4' partition 5 - change '5' partition 6 - change '6' partition 7 - change '7' partition select - select a predefined table modify - modify a predefined partition table name - name the current table print - display the current table label - write partition map and label to the disk !
- execute , then return quit partition> print Current partition table (original): Total disk cylinders available: 9419 + 2 (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 unassigned wm 0 0 (0/0/0) 0 1 unassigned wm 0 0 (0/0/0) 0 2 backup wm 0 - 19453 149.03GB (19454/0/0) 312528510 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 8 boot wm 0 - 0 7.84MB (1/0/0) 16065 9 unassigned wm 0 0 (0/0/0) 0 partition>
We need to ensure slice 0 is configured, in my case I configure it to be the whole size of the disk, also ensure disk is labeled :
partition> 0 Part Tag Flag Cylinders Size Blocks 0 unassigned wm 0 0 (0/0/0) 0 Enter partition id tag[unassigned]: root Enter partition permission flags[wm]: wm Enter new starting cyl: 1 Enter partition size[0b, 0c, 0e, 0mb, 0gb]: 19453c partition> print Current partition table (original): Total disk cylinders available: 9419 + 2 (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm 1 - 19453 149.02GB (19453/0/0) 312512445 1 unassigned wm 0 0 (0/0/0) 0 2 backup wm 0 - 19453 149.03GB (19454/0/0) 312528510 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 8 boot wm 0 - 0 7.84MB (1/0/0) 16065 9 unassigned wm 0 0 (0/0/0) 0 partition> label Ready to label disk, continue? y partition> quit
Again ensure disk is definitely labeled from the format prompt :
$ format> label Ready to label disk, continue? y
Choose label from the format(1) menu, and select "y". If prompted for specific label type, ensure to choose SMI. Once labeled choose quit to get back to shell prompt.
$ format> quit
3. Attach USB Disk To Main Pool Creating Mirror
Now to attach the new USB drive to the existing rpool and thus creating a mirror. This time ensure you use s0 (slice 0) at end of device.
$ pfexec zpool attach -f rpool c5t0d0s0 c6t0d0s0
Once this completes run zpool status again, and you will see the new device as part of your new mirror configuration, and will also show percentage re-silvering. It took 1hr 45mins to resilver a 200GB disk, and 120GB of data.
$ zpool status pool: rpool state: ONLINE config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c5t0d0s0 ONLINE 0 0 0 c6t0d0s0 ONLINE 0 0 0 1.18G resilvered errors: No known data errors
4. Install Grub on USB Disk
Finally we need to install grub on the USB disk to ensure it is bootable :
$ pfexec installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c6d0s0
5. Some Caveats
Whilst I definitely think this is a great backup solution, it doesn't come without some caveats/kinks that I've come across over the first few days with this setup.
- Ensure your USB disk is at least as big as your laptop drive. In a ZFS mirror configuration, the smallest disk dictates the size of the pool.
- Always have your USB drive connected a boot time. For some reason Solaris will not boot unless your drive is connected at boot time. This to me seems like a bug.
- When shutting down, remember the exact USB port your USB drive was connected to, on some occasions I've noticed Solaris would not boot unless the USB drive was connected to the exact same port when shutdown.
- NEVER shutdown whilst your USB drive is on offline mode. You can take your USB drive offline via zfs offline pool disk, however if you shutdown whilst it is offline (and disconnected) Solaris simply hangs when attempting to boot, again another possible bug. To recover from this situation I had to do the following :
- Boot into LiveCD environment of a Solaris build of the same or more recent than currently installed build on your laptop. Ensure USB drive is connected when booting.
- Once booted import your pool, and bring USB drive back online :
$ zpool import -f rpool $ zpool online rpool %ltusbdisk%gt
- Now reboot and you should be good to go.