Wednesday Feb 18, 2009

Rocks Cluster Customization Part 1

Rocks is a great HPC management software that allow very quick and easy deployment of HPC clusters. If using the default configuration and software packages that comes with Rocks, a typical average size cluster can be ready in no more than a few day. Besides doing bare metal provisioning of OS, additional software packaged as Rocks rolls can be added to the base Rocks distribution. Once these rolls are installed, provisioning a compute node will also automatically deploy and configure the service.

Rocks uses a their own way of assign hostnames. The suggested order of provisioning compute nodes is to start from first rack, from top server to bottom server. The default hostname are in the form of "compute-x-y", where x is the rack number and y is the position of the node in the rack. For example, the top most node in the first rack will be compute-0-0, the second node compute-0-1 and so on. The nodes in the next rack will then be compute-1-0, compute-1-1...

Although this works well for most, but definitely not for all. There are several ways to change the hostnames depending on how customized you want it to be. If you want to keep the numbering scheme but just want to change the prefix (e.g. to mynode-0-0), what you can do is to add a new appliance.

$ rocks add appliance mynode membership="My Compute Node" node=compute

What the command does is to add a new appliance using the same compute configuration. Then when provisioning new nodes, in the insert-ethers main menu, select "My Compute Node" appliance. When the new node is provisioned, it will be assigned hostname like mynode-x-y.

If you want to even go to the extend of changing the hostname numbering, what you can do is to fix the hostname and IP address in the Rocks database.

$ rocks add host n00 membership=compute rack=1 rank=1
$ rocks add host n01 membership=compute rack=1 rank=2
...
$ rocks add host n08 membership=compute rack=1 rank=8

In addition, if you want to assign the IP addresses manually:

$ rocks add host interface n00 iface=eth0 ip=10.100.1.8 mac=00:11:22:33:44:55:GG subnet=private name=n00
$ rocks add host interface n01 iface=eth0 ip=10.100.1.9 mac=00:11:22:33:44:55:FF subnet=private name=n01
...

Or if you want to configure a 2nd interface (e.g. ib0), you can add a new network and configure the interface in the Rocks database.

$ rocks set network IB subnet=10.200.0.0 netmask=255.255.0.0
$ rocks add host interface n00 iface=ib0 ip=10.200.1.8 mac=00:11:22:33:44:55:AA subnet=IB name=n00-ib
$ rocks add host interface n01 iface=ib0 ip=10.200.1.8 mac=00:11:22:33:44:55:BB subnet=IB name=n01-ib
...

After the changes are made, make sure to sync the configurations.

$ rocks sync config
$ make -C /var/411 #use "make -C /var/411 force" if necessary, but it's slower
$ insert-ethers --update

About

Melvin Koh

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today