News, tips, partners, and perspectives for the Oracle Solaris operating system
Prior to the 4.4 release of Oracle Solaris Cluster (OSC), it was not possible to perform automatic public network configuration for Exclusive-IP Zone Cluster (ZC) by specifying a System Configuration (SC) profile to the clzonecluster 'install' command.
To illustrate this let us consider installation of a typical ZC with a separate IP stack and two data-links to achieve network redundancy needed to run HA services. The data-links which are vnics previously created in the global zone are configured as part of an IPMP group that is needed to host the LogicalHostname or SharedAddress resource IP address. The zone cluster was configured as shown by the clzc 'export' command output below.
root@clusterhost1:~# clzc export zc1 create -b set zonepath=/zones/zc1 set brand=solaris set autoboot=false set enable_priv_net=true set enable_scalable_svc=false set file-mac-profile=none set ip-type=exclusive add net set address=192.168.10.10 set physical=auto end add attr set name=cluster set type=boolean set value=true end add node set physical-host=clusterhost1 set hostname=zc1-host-1 add net set physical=vnic3 end add net set physical=vnic0 end add privnet set physical=vnic1 end add privnet set physical=vnic2 end end add node set physical-host=clusterhost2 set hostname=zc1-host-2 add net set physical=vnic3 end add net set physical=vnic0 end add privnet set physical=vnic1 end add privnet set physical=vnic2 end end
In OSC 4.3, after installing the ZC with a SC profile and booting it up, ZC will be in Online Running state but without the public network configuration. The following ipadm(1M) commands are needed to set up the static network configuration in each non-global zone of the ZC.
root@zc1-host-1:~# ipadm create-ip vnic0 root@zc1-host-1:~# ipadm create-ip vnic3 root@zc1-host-1:~# ipadm create-ipmp -i vnic0 -i vnic3 sc_ipmp0 root@zc1-host-1:~# ipadm create-addr -T static -a 192.168.10.11/24 sc_ipmp0/v4
root@zc1-host-2:~# ipadm create-ip vnic0 root@zc1-host-2:~# ipadm create-ip vnic3 root@zc1-host-2:~# ipadm create-ipmp -i vnic0 -i vnic3 sc_ipmp0 root@zc1-host-2:~# ipadm create-addr -T static -a 192.168.10.12/24 sc_ipmp0/v4
In OSC 4.4 it is now possible to build a SC profile such that no manual steps will be required to complete the network configuration and all the zones of the ZC can boot up to "Online Running" state upon first boot of the ZC.
How is this made possible in OSC 4.4 on Solaris 11.4?
Well, the clzonecluster(8CL) command can recognize sections of the SC profile XML that are applicable for individual zones of the ZC by inserting these sections within the <instances_for_node node_name="ZCNodeName"></instances_for_node> XML tags. Other sections of the SC profile that are not within these XML tags are applicable for all the zones of the ZC.
Solaris 11.4 now supports arbitrarily complex network configurations in SC profiles. The following is a snippet of the SC profile that can be used for our typical ZC configuration that is derived from the template /usr/share/auto_install/sc_profiles/ipmp_network.xml. The section of the SC profile which is common for all the zones of the ZC has not been included in this snippet.
<instances_for_node node_name="zc1-host-1">
<service version="1" name="system/identity">
<instance enabled="true" name="node">
<property_group name="config">
<propval name="nodename" value="zc1-host-1"/>
</property_group>
</instance>
</service>
<service name="network/ip-interface-management" version="1" type="service">
<instance name="default" enabled="true">
<property_group name="interfaces" type="application">
<!-- vnic0 interface configuration -->
<property_group name="vnic0" type="interface-ip">
<property name="address-family" type="astring">
<astring_list>
<value_node value="ipv4"/>
<value_node value="ipv6"/>
</astring_list>
</property>
<propval name="ipmp-interface" type="astring" value="sc_ipmp0"/>
</property_group>
<!-- vnic3 interface configuration -->
<property_group name="vnic3" type="interface-ip">
<property name="address-family" type="astring">
<astring_list>
<value_node value="ipv4"/>
<value_node value="ipv6"/>
</astring_list>
</property>
<propval name="ipmp-interface" type="astring" value="sc_ipmp0"/>
</property_group>
<!-- IPMP interface configuration -->
<property_group name="sc_ipmp0" type="interface-ipmp">
<property name="address-family" type="astring">
<astring_list>
<value_node value="ipv4"/>
<value_node value="ipv6"/>
</astring_list>
</property>
<property name="under-interfaces" type="astring">
<astring_list>
<value_node value="vnic0"/>
<value_node value="vnic3"/>
</astring_list>
</property>
<!-- IPv4 static address -->
<property_group name="data1" type="address-static">
<propval name="ipv4-address" type="astring" value="192.168.10.11"//>
<propval name="prefixlen" type="count" value="24"/>
<propval name="up" type="astring" value="yes"/>
</property_group>
</property_group>
</property_group>
</instance>
</service>
<instances_for_node node_name="zc1-host-2">
<service version="1" name="system/identity">
<instance enabled="true" name="node">
<property_group name="config">
<propval name="nodename" value="zc1-host-2"/>
</property_group>
</instance>
</service>
<service name="network/ip-interface-management" version="1" type="service">
<instance name="default" enabled="true">
<property_group name="interfaces" type="application">
<!-- vnic0 interface configuration -->
<property_group name="vnic0" type="interface-ip">
<property name="address-family" type="astring">
<astring_list>
<value_node value="ipv4"/>
<value_node value="ipv6"/>
</astring_list>
</property>
<propval name="ipmp-interface" type="astring" value="sc_ipmp0"/>
</property_group>
<!-- vnic0 interface configuration -->
<property_group name="vnic3" type="interface-ip">
<property name="address-family" type="astring">
<astring_list>
<value_node value="ipv4"/>
<value_node value="ipv6"/>
</astring_list>
</property>
<propval name="ipmp-interface" type="astring" value="sc_ipmp0"/>
</property_group>
<!-- IPMP interface configuration -->
<property_group name="sc_ipmp0" type="interface-ipmp">
<property name="address-family" type="astring">
<astring_list>
<value_node value="ipv4"/>
<value_node value="ipv6"/>
</astring_list>
</property>
<property name="under-interfaces" type="astring">
<astring_list>
<value_node value="vnic0"/>
<value_node value="vnic3"/>
</astring_list>
</property>
<!-- IPv4 static address -->
<property_group name="data1" type="address-static">
<propval name="ipv4-address" type="astring" value="192.168.10.12"//>
<propval name="prefixlen" type="count" value="24"/>
<propval name="up" type="astring" value="yes"/>
</property_group>
</property_group>
</property_group>
</instance>
</service>
</instances_for_node>
You can find the complete SC profile here. Some fields like zone host name, IP and encrypted passwords needs to be substituted in this file. Other changes can be made to this profile for different configuration for e.g, configuring an Active-Standby IPMP group instead of Active-Active IPMP configuration shown in this example.
We can use this SC profile to install and boot our ZC as shown below.
root@clusterhost1:~# clzc install -c /var/tmp/zc1_config.xml zc1 Waiting for zone install commands to complete on all the nodes of the zone cluster "zc1"...
root@clusterhost1:~# clzc boot zc1 Waiting for zone boot commands to complete on all the nodes of the zone cluster "zc1"...
After a short duration, we will see the ZC in Online Running state.
root@clusterhost1:~# clzc status zc1 === Zone Clusters === --- Zone Cluster Status --- Name Brand Node Name Zone Host Name Status Zone Status ---- ----- --------- -------------- ------ ----------- zc1 solaris clusterhost1 zc1-host-1 Online Running clusterhost2 zc1-host-2 Online Running
root@zc1-host-1:~# ipadm NAME CLASS/TYPE STATE UNDER ADDR clprivnet1 ip ok -- -- clprivnet1/? static ok -- 172.16.3.65/26 lo0 loopback ok -- -- lo0/v4 static ok -- 127.0.0.1/8 lo0/v6 static ok -- ::1/128 sc_ipmp0 ipmp ok -- -- sc_ipmp0/data1 static ok -- 192.168.10.11/24 vnic0 ip ok sc_ipmp0 -- vnic1 ip ok -- -- vnic1/? static ok -- 172.16.3.129/26 vnic2 ip ok -- -- vnic2/? static ok -- 172.16.3.193/26 vnic3 ip ok sc_ipmp0 --
root@zc1-host-2:~# ipadm NAME CLASS/TYPE STATE UNDER ADDR clprivnet1 ip ok -- -- clprivnet1/? static ok -- 172.16.3.66/26 lo0 loopback ok -- -- lo0/v4 static ok -- 127.0.0.1/8 lo0/v6 static ok -- ::1/128 sc_ipmp0 ipmp ok -- -- sc_ipmp0/data1 static ok -- 192.168.10.12/24 vnic0 ip ok sc_ipmp0 -- vnic1 ip ok -- -- vnic1/? static ok -- 172.16.3.130/26 vnic2 ip ok -- -- vnic2/? static ok -- 172.16.3.194/26 vnic3 ip ok sc_ipmp0 --
With this feature it is easy to create templates for SC profiles and can be used for fast deployments of ZC's without the need for administrator intervention to complete system configuration of the zones in the ZC.
For more details on the SC profile configuration in 11.4 refer to Customizing Automated Installations With Manifests and Profiles.
For cluster documentation refer to clzonecluster(8cl).