Solaris Cluster on a laptop using VirtualBox, iSCSI and a quorum server

I was intrigued by Gordon Johnston's blog article about creating a 2 node Cluster Express cluster using VirtualBox, and I decided I had to try it.

Here's the final result:

(You can click on the image for a fullsized picture.)

It shows a VNC connection to my laptop which has 4 windows up: The 2 VirtualBox instances showing cxnode1 and cxnode2, a terminal window on the laptop itself, showing the quorum configuration and the iSCSI share, and in the background, the VirtualBox console showing the network configuration of cxnode1. Finally, in the main screen, I'm running Sun Cluster Manager and connected to the cluster, showing the topology and the Resource Group configuration of apache.

Rather than simply duplicating what Gordon did, I decided to go with Solaris Cluster 3.2 U1 as my cluster software. This meant that I had to use an internally available package of 32 bit kernel drivers, as SC 3.2 is shipped as a 64bit only product. (Cluster Express has both 32 and 64 bit variants.) The reason for going with Solaris Cluster is that there are instances where I want to duplicate a scenario that a customer is seeing, and in most cases, that's on Solaris Cluster.

I'm hesitant to duplicate Gordon's article, as I pretty much followed his guide, so I urge you to read his article as it is quite detailed, I'll only highlight where I changed things.

  1.  I only really wanted a working cluster, went for the simpler set up of HA Apache
  2. Cluster node O/Ss would be Solaris 10 05/08
  3. Host O/S is Solaris Nevada Build 93 (as that was what was already running on my laptop)
  4. I wanted to Jumpstart the nodes using JET.

Preparing your physical host

I already had VirtualBox 1.6.2 installed, so there wasn't much to do there. The main trick was configuring my laptop to allow Host Interface networking. This bit is Solaris specific. You basically need to do 2 things.

Firstly, you need to set up a bunch of virtual nics, and assign MAC addresses to them. As this needs to be redone on every reboot, I followed Dave Tong's advice  and set up an rc3.d script. (and no, I didn't bother to use SMF either, you can bite me too if you want.)

#!/sbin/sh
state="$1"
case "${state}" in
'start')
    echo 'Plumbing VNICs'
    for MAC in 10 20 30 40
    do
        VNIC=`/usr/lib/vna nge0 c0:ff:ee:0:0:$MAC`
        ifconfig $VNIC plumb
    done
    ;;
'stop')
    ;;
\*)
   echo "Usage: $0 { start | stop }"
    exit 1
    ;;
esac

(I did use Gordon's cute fake MAC address c0:ff:ee 'cos its easy to remember)

Secondly, you REALLY REALLY need to make sure that you give the VBoxSVC privileges to configure the interface, otherwis your virtual machines will fail when starting. Whenever you start VirtualBox, you need to get the PID of the VBoxSVC, and run: ppriv -s AEI+net_rawaccess <pid>

Creating your Virtual hosts

Basically I followed Gordon's instructions for this, with TWO exceptions. I needed to select Network in the Boot Order menu in Settings-->General-->Advanced as PXE is not enabled by default. I also found that I couldn't do a PXE boot if I used the Intel driver, so I used the PC-net FAST III driver throughout which worked fine.

Installing Solaris on the Virtual hosts

I've already got JET installed on my laptop, so preparing to install Solaris was an amazingly simple process:

  1. /opt/SUNWjet/bin/make_template cxnode1 base_config
  2. Edit the template
  3. /opt/SUNWjet/bin/make_client cxnode1
  4. Start the cxnode1 Virtual Machine, Press F12, Select Lan Boot, and watch it install.
  5. /opt/SUNWjet/bin/make_template -T cxnode1 cxnode2
  6. Edit the cxnode2 template and change 3 variables.
  7. /opt/SUNWjet/bin/make_client cxnode2
  8. Start the cxnode2 Virtual Machine. Press F12, Select Lan Boot and watch it install.

Of course, the devil is in the detail, I'd already configured the Solaris media, and these are the changes I made to a default template for the build to work:

base_config_ClientArch=i86pc
\*base_config_ClientEther=c0:ff:ee:0:0:10
base_config_ClientOS=10_0508x
base_config_client_allocation="dhcp"
\*base_config_sysidcfg_ip_address=192.168.1.201
base_config_sysidcfg_netmask=255.255.255.0
base_config_profile_fdisk="alldisk"
base_config_profile_cluster=SUNWCXall
base_config_profile_root=free 
base_config_profile_swap=2048
base_config_profile_s6_mtpt="/globaldevices"
base_config_profile_s6_
base_config_ipmp_networkifs="pcn0_pcn1"
\*base_config_ipmp_networkif_pcn0_pcn1="prod sp 192.168.1.211 192.168.1.212 24 cxnode1 192.168.1.201"

(the ones with \* are the variables I had to update when I created the cxnode2 template) 

Installing Solaris Cluster on the nodes

As the nodes were running in 32bit mode, (VirtualBox 1.7 should have 64bit support, which means this part won't be necessary in the future) I needed to use an internally available set of 32 bit kernel drivers in a packages called SUNWscka. Basically, install sc3.2 as usual using the installer, but don't configure anything, and don't configure a quorum device. Then pkgadd SUNWscka on both nodes, and reboot them both.

The rest

After that, I just followed Gordon's instructions for setting up the Quorum server, setting up the iSCSI devices on the hosts, mounting them on the cluster nodes, and creating a zfs pool using them.

Sadly, this now means I can play with and configure all sorts of things in a Sun Cluster environment from the comfort of my laptop!

tags:

Comments:

Post a Comment:
Comments are closed for this entry.
About

mramcha

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today