moving from nevada and live upgrade to opensolaris

so recently i made the plunge into running opensolaris. previously both my desktop and laptop were running nevada and i was using live upgrade to update the systems every two weeks to the latest build. on my desktop i had four live upgrade boot environments and on my laptop i had three. i also had zpools with lots of data on both systems.

ideally i wanted to be able to install opensolaris into an existing slice on my disks that was currently in use as a live upgrade boot environment. this would allow me to preserve my existing zpools and not have to backup and restore my entire system. it would also mean that i'd have my other nevada BE's left around in case i needed to go back to them for some reason. unfortunately, i discovered that the new opensolaris installer will only install to an entire disk or a partition, installing to an existing slice is not an option.

so on my laptop i underwent the time consuming process of backing up and restoring all my zpool data via zfs send/receive -R to an external usb hard drive, and doing a full fresh install. for my desktop i decided to try a different approach. i decided to install opensolaris.org manually into an existing slice while running nevada. the rest of this blog entry will document how i did this.

<disclaimer> i'm documenting this because others have expressed an interest in knowing how to do this. that said, this process should not be attempted by the faint of heart and this takes you pretty far off the "supported" reservation. (i'm still running into bugs that no one else has seen because no one else installs systems this way.) while this seems to have worked out fine for me, ymmv. </disclaimer>

Step 1 - free up a slice and create a zpool to install to. to do this i deleted an existing live upgrade BE and created a zpool on that slice. i also made sure to manually set all the zfs properties that the opensolaris installer seems to set.

    # free up a slice from an existing BE
    lufslist disk1a_nv_91 | grep ' / '
    ludelete disk1a_nv_91
    
    # create and configure a zpool on that slice
    zpool create -f rpool c1t1d0s0
    zfs set compression=on rpool
    zfs create -p rpool/ROOT/opensolaris/opt
    zfs set canmount=noauto rpool/ROOT/opensolaris
    zfs set canmount=noauto rpool/ROOT/opensolaris/opt
    zfs set mountpoint=legacy rpool/ROOT/opensolaris
    zfs set mountpoint=legacy rpool/ROOT/opensolaris/opt
    

Step 2 - install a copy of ips. you need ips to install opensolaris. normally when you boot the opensolaris cd, you have ips, but i needed to install a copy in nevada. the good news is that an ips gate build creates sysv packages that allow ips to be installed on nevada. the bad news is that pre-built packages are not yet available for download. so i just pulled a copy of the latest version of ips from the ips gate and built it. you need a pretty recent version of nevada todo this (i was running snv_91).

    # setup misc environment variables
    WS=/tmp/pkg
    PATH=$WS/gate/proto/root_`uname -p`/usr/bin:$PATH
    PYTHONPATH=$WS/gate/proto/root_`uname -p`/usr/lib/python2.4/vendor-packages
    export WS PATH PYTHONPATH
    
    # download and build the ips source in /tmp/pkg
    mkdir $WS
    cd $WS
    hg clone ssh://anon@hg.opensolaris.org/hg/pkg/gat e
    cd gate/src
    make install
    

Step 3 - install an opensolaris image into our zpool. i figured out how to do this by talking to folks on the ips team and also by looking at the ips brand zone install script (which is at /usr/lib/brand/ipkg/pkgcreatezone). some of the things that i needed todo are pretty hacky and non-obvious and in may represent bugs in the existing system that should be fixed.

    # we're going to do our install in /a
    PKG_IMAGE=/a; export PKG_IMAGE
    
    # mount our zpool on /a
    mkdir -p $PKG_IMAGE
    mount -F zfs rpool/ROOT/opensolaris $PKG_IMAGE
    mkdir -p $PKG_IMAGE/opt
    mount -F zfs rpool/ROOT/opensolaris/opt $PKG_IMAGE/opt
    
    # create the basic opensolaris install image.
    pkg image-create -F -a opensolaris.org=http://pkg.opensolaris.org $PKG_IMAGE
    pkg refresh
    pkg install entire@0.5.11-0.93
    pkg install SUNWcsd SUNWcs
    pkg install redistributable
    
    # seed the initial smf repository
    cp $PKG_IMAGE/lib/svc/seed/global.db $PKG_IMAGE/etc/svc/repository.db
    chmod 0600 $PKG_IMAGE/etc/svc/repository.db
    chown root:sys $PKG_IMAGE/etc/svc/repository.db
    
    # setup smf profiles
    ln -s ns_files.xml $PKG_IMAGE/var/svc/profile/name_service.xml
    ln -s generic_limited_net.xml $PKG_IMAGE/var/svc/profile/generic.xml
    ln -s inetd_generic.xml $PKG_IMAGE/var/svc/profile/inetd_services.xml
    ln -s platform_none.xml $PKG_IMAGE/var/svc/profile/platform.xml
    
    # mark the new system image as uninstalled
    sysidconfig -b $PKG_IMAGE -a /lib/svc/method/sshd
    touch $PKG_IMAGE/etc/.UNCONFIGURED
    
    # configure our new /etc/vfstab
    printf "rpool/ROOT/opensolaris -\\t/\\t\\tzfs\\t-\\tno\\t-\\n" >> $PKG_IMAGE/etc/vfstab
    chmod a+r $PKG_IMAGE/etc/vfstab
    
    # carry over our curent swap settings to the new image
    grep swap /etc/vfstab | grep -v \^swap >> $PKG_IMAGE/etc/vfstab
    
    # turn off root as a role
    print "/\^root::::type=role;\\ns/\^root::::type=role;/root::::/\\nw" |\\
        ed -s $PKG_IMAGE/etc/user_attr
    
    # delete the "jack" user
    print "/\^jack:/d\\nw" | ed -s $PKG_IMAGE/etc/passwd
    chmod u+w $PKG_IMAGE/etc/shadow
    print "/\^jack:/d\\nw" | ed -s $PKG_IMAGE/etc/shadow
    chmod u-w $PKG_IMAGE/etc/shadow
    
    # copy of my existing host sshd keys
    cp -p /etc/ssh/\*key\* $PKG_IMAGE/etc/ssh
    
    # configure /dev in the new image
    devfsadm -R $PKG_IMAGE
    ln -s ../devices/pseudo/sysmsg@0:msglog $PKG_IMAGE/dev/msglog
    
    # update the boot archive in the new image
    bootadm update-archive -R $PKG_IMAGE
    
    # workaround 6717352
    chmod a-x $PKG_IMAGE/usr/lib/gvfsd-trash
    
    # workaround 6728023
    rm -f $PKG_IMAGE/etc/gtk-2.0/gdk-pixbuf.loaders
    rm -f $PKG_IMAGE/etc/64/gtk-2.0/gdk-pixbuf.loaders
    

Step 4 - modify grub so we can boot the new image. at this point the new image is pretty much ready to boot. all we need todo is configure grub so that it can boot the new image. unfortunately, this is probably the trickiest step in this entire process, since if you screw it up you could lose your grub menu, which makes it difficult to boot any boot environment. so here's what i did:

    # update to the latest version of grub (this command generated
    # some errors which i ignored).
    $PKG_IMAGE/boot/solaris/bin/update_grub -R $PKG_IMAGE
    
    # create an informational grub menu in the install image that
    # points us to the real grub menu.
    cat <<-EOF > $PKG_IMAGE/boot/grub/menu.lst
    
    #########################################################################
    #                                                                       #
    # For zfs root, menu.lst has moved to /rpool/boot/grub/menu.lst.        #
    #                                                                       #
    #########################################################################
    
    EOF
    
    # create the new real grub menu
    cat <<-EOF > /rpool/boot/grub/menu.lst2
    default 0
    timeout 10
    splashimage /boot/grub/splash.xpm.gz
    
    title Solaris 2008.11 snv_93 X86
    findroot (pool_rpool,0,a)
    bootfs rpool/ROOT/opensolaris
    kernel\\$ /platform/i86pc/kernel/\\$ISADIR/unix -k -B \\$ZFS-BOOTFS,console=ttya
    module\\$ /platform/i86pc/\\$ISADIR/boot_archive
    
    title Solaris xVM
    findroot (pool_rpool,0,a)
    bootfs rpool/ROOT/opensolaris
    kernel\\$ /boot/\\$ISADIR/xen.gz com1=9600,8n1 console=com1
    module\\$ /platform/i86xpv/kernel/\\$ISADIR/unix /platform/i86xpv/kernel/\\$ISADIR/unix -k -B \\$ZFS-BOOTFS
    module\\$ /platform/i86pc/\\$ISADIR/boot_archive
    EOF
    
    # make the grub menu files readable by everyone.
    chmod a+r $PKG_IMAGE/boot/grub/menu.lst
    chmod a+r /rpool/boot/grub/menu.lst
    
    # setup /etc/bootsign so that grub can find this zpool
    mkdir -p /rpool/etc
    echo pool_rpool > /rpool/etc/bootsign
    

Step 5 - cross your fingers and reboot. before doing this, you might want to copy over your existing grub menu entries to the new grub menu file at /rpool/boot/grub/menu.lst. once you're ready to reboot into opensolaris, you can do the following:

    umount $PKG_IMAGE/opt
    umount $PKG_IMAGE
    reboot
    
Comments:

Post a Comment:
Comments are closed for this entry.
About

edp

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today