News, tips, partners, and perspectives for the Oracle Solaris operating system
When using the Automated Installer, you define system configurations in System Configuration (SC) profile files. SMF loads these SC-profiles into its database during first boot of a freshly installed system. The network/install service, which existed before Sorlaris 11.4, initializes the system’s network configuration from its SMF based configuration parameters. This service only supports creation of IP addresses over the system's existing physical interfaces. There are a great many other Solaris network features commonly used in data centers such as jumbo frames, link aggregation, vlans, and flows that can not be configured by the network/install service. Now that the dladm(8), flowadm(8), ipadm(8), netcfg(8), and route(8) utilities store their persistent information in SMF, you can directly configure network services in SC-profiles to handle anything these utilities support.
SC-profiles use an XML based format to define service configuration information. If your like me, you would rather avoid the effort it takes to learn the detailed syntax rules for network configuration. Fortunately, you don't have to. I find an easier approach is to use the network utilities to set up the configuration I want on a live staging system. I use the SMF's svccfg(8) utility to capture the running configuration in XML format, then create my SC-profiles directly from the svccfg generated files.
The instructions below show how I built SC-profiles to automatically install SPARC systems with a pair of high speed ethernet links supporting 9000 byte jumbo frames in a link aggregation. The process works exactly the same on X86 systems.
Step 1: Install staging system without external interfaces configured
I started by installing the staging system with no external network interfaces configured. Solaris automatically configures the following on first boot after the install completes.
- dladm link name to physical device mappings
- loopback IP interface (lo0)
- IP interface to the service processor (sp-phys)
These details could cause unwanted problems if they were included in my final SC-profile. For example, physical device names are unlikely to be the same on all systems where I expect to apply my custom network configuration. I'll be capturing this initial network configuration without my changes to identify what parts of the full configuration I should remove later.
Do the following on the AI Server.
Use a text editor to create the AI manifest file. The manifest in this example installs the large server group from a solaris repository. Replace "http://pkg.oracle.com/solaris/release/" with a local copy of the solaris repository if one is available. Feel free to list any other packages desired.
base-manifest.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE auto_install SYSTEM "file:///usr/share/install/ai.dtd.1">
<auto_install>
<ai_instance name="lab" auto_reboot="true">
<target>
<logical>
<zpool name="rpool" is_root="true">
<filesystem name="export" mountpoint="/export"/>
<filesystem name="export/home"/>
<be name="solaris"/>
</zpool>
</logical>
</target>
<software type="IPS">
<source>
<publisher name="solaris">
<origin name="http://pkg.oracle.com/solaris/release/"/>
</publisher>
</source>
<software_data action="install">
<name>pkg:/entire@latest</name>
<name>pkg:/group/system/solaris-large-server</name>
</software_data>
</software>
</ai_instance>
</auto_install>
Use sysconfig(8) to generate the initial SC-profile file with login, language, and timezone information you normally used at the site. Be sure to select "No network" on the initial Network Configuration screen. This example used the Computer Name "headbanger" for the example staging system.
# sysconfig create-profile -o headbanger
(fill in sysconfig's on-screen menus)
# mv headbanger/sc_profile.xml headbanger/base-profile.xml
Use installadm(8) to configure the install server. The staging system in this example is a SPARC system with MAC address 0:0:5e:0:53:d2.
# installadm create-service -y -n nettest-sparc \
-a sparc \
-p solaris=http://pkg.oracle.com/solaris/release/ \
-s install-image/solaris-auto-install@latest
# installadm create-manifest -n nettest-sparc \
-d -m base-manifest \
-f base-manifest.xml
# installadm create-profile -n nettest-sparc \
-p base-profile \
-f base-profile.xml
# installadm create-client -n nettest-sparc \
-e 0:0:5e:0:53:d2
Do the following on the staging system.
Log into the system's service processor, connect to the system console, and start a network boot to launch the installation.
-> start /SP/console
(halt operating system if currently running)
{0} ok boot net:dhcp - install-> start /SP/console
When first boot completes, verify no external interfaces have been configured yet.
root@headbanger:~# dladm
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net2 phys 1500 up --
net3 phys 1500 up --
sp-phys0 phys 1500 up --
root@headbanger:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
lo0 loopback ok -- --
lo0/v4 static ok -- 127.0.0.1/8
lo0/v6 static ok -- ::1/128
sp-phys0 ip ok -- --
sp-phys0/v4 static ok -- 203.0.113.77/24
Capture the initial first-boot configuration in SC-profile format for later reference.
root@headbanger:~# svccfg extract -l admin \
network/datalink-management:default \
> initial-profile.xml
root@headbanger:~# svccfg extract -l admin \
network/ip-interface-management:default \
>> initial-profile.xml
Step 2: Configure network on staging system using Solaris utilities
Use Solaris network configuration utilities to create the desired configuration.
root@headbanger:~# dladm set-linkprop -p mtu=9000 net0
root@headbanger:~# dladm set-linkprop -p mtu=9000 net1
root@headbanger:~# dladm create-aggr -m dlmp -l net0 -l net1 aggr0
root@headbanger:~# ipadm create-ip aggr0
root@headbanger:~# ipadm create-addr -a 198.51.100.63/24 aggr0
aggr0/v4
root@headbanger:~# ipadm create-addr -T addrconf \
-p stateless=no,stateful=no aggr0 aggr0/v6
root@headbanger:~# ipadm create-addr \
-a 2001:db8:414:60bc::3f/64 aggr0 aggr0/v6a
root@headbanger:~# route -p add default 198.51.100.1
add net default: gateway 198.51.100.1
root@headbanger:~# dladm set-linkprop \
-p probe-ip=198.51.100.63+198.51.100.1 aggr0
Verify the configuration was defined as intended.
root@headbanger:~# dladm
LINK CLASS MTU STATE OVER
aggr0 aggr 9000 up net0 net1
net0 phys 9000 up --
net1 phys 9000 up --
net2 phys 1500 up --
net3 phys 1500 up --
sp-phys0 phys 1500 up --
root@headbanger:~# dladm show-linkprop -p mtu
LINK PROPERTY PERM VALUE EFFECTIVE DEFAULT POSSIBLE
aggr0 mtu rw 9000 9000 1500 576-9706
net0 mtu rw 9000 9000 1500 576-9706
net1 mtu rw 9000 9000 1500 576-9706
net2 mtu rw 1500 1500 1500 576-9706
net3 mtu rw 1500 1500 1500 576-9706
sp-phys0 mtu rw 1500 1500 1500 1500
root@headbanger:~# dladm show-aggr -Sn
LINK PORT FLAGS STATE TARGETS XTARGETS
aggr0 net0 u--3 active 198.51.100.1 net1
-- net1 u-2- active -- net0
root@headbanger:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
aggr0 ip ok -- --
aggr0/v4 static ok -- 198.51.100.63/24
aggr0/v6 addrconf ok -- fe80::8:20ff:fe67:6164/10
aggr0/v6a static ok -- 2001:db8:414:60bc::3f/64
lo0 loopback ok -- --
lo0/v4 static ok -- 127.0.0.1/8
lo0/v6 static ok -- ::1/128
sp-phys0 ip ok -- --
sp-phys0/v4 static ok -- 203.0.113.77/24
root@headbanger:~# netstat -nr
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
default 198.51.100.1 UG 2 31
198.51.100.0 198.51.100.63 U 3 73854 aggr0
127.0.0.1 127.0.0.1 UH 4 68 lo0
203.0.113.0 203.0.113.77 U 3 131889 sp-phys0
Routing Table: IPv6
Destination/Mask Gateway Flags Ref Use If
------------------------- --------------------------- ----- --- ------- -----
::1 ::1 UH 2 24 lo0
2001:db8:414:60bc::/64 -- U 2 0 aggr0
2001:db8:414:60bc::/64 2001:db8:414:60bc::3f U 2 0 aggr0
fe80::/10 fe80::8:20ff:fe67:6164 U 3 0 aggr0
default fe80::200:5eff:fe00:530c UG 2 0 aggr0
Reboot the staging system and retest to ensure the configuration was persistently applied.
Step 3: Capture the custom network configuration from SMF
Use the SMF svccfg(8) utility to extract the current configuration from the modified network services.
root@headbanger:~# svccfg extract -l admin \
network/datalink-management:default \
>> labnet-profile.xml
root@headbanger:~# svccfg extract -l admin \
network/ip-interface-management:default \
>> labnet-profile.xml
Edit the labnet-profile.xml file to:
- Delete replicated service bundle overhead between the extracted information of each service.
- Delete entries that exist in the initial-profile.xml file saved previously in Step 1.
labnet-profile.xml
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='profile' name='labnet-profile'>
<service name='network/datalink-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='datalinks' type='application'>
<property_group name='aggr0' type='datalink-aggr'>
<propval name='aggr-mode' type='astring' value='dlmp'/>
<propval name='force' type='boolean' value='false'/>
<propval name='key' type='count' value='0'/>
<propval name='lacp-mode' type='astring' value='off'/>
<propval name='lacp-timer' type='astring' value='short'/>
<propval name='media' type='astring' value='Ethernet'/>
<propval name='num-ports' type='count' value='2'/>
<propval name='policy' type='astring' value='L4'/>
<propval name='probe-ip' type='astring' value='198.51.100.63+198.51.100.1'/>
<property name='ports' type='astring'>
<astring_list>
<value_node value='net0'/>
<value_node value='net1'/>
</astring_list>
</property>
</property_group>
<property_group name='net0' type='datalink-phys'>
<propval name='devname' type='astring' value='i40e0'/>
<propval name='loc' type='astring' value='/SYS/MB'/>
<propval name='media' type='astring' value='Ethernet'/>
<propval name='mtu' type='count' value='9000'/>
</property_group>
<property_group name='net1' type='datalink-phys'>
<propval name='devname' type='astring' value='i40e1'/>
<propval name='loc' type='astring' value='/SYS/MB'/>
<propval name='media' type='astring' value='Ethernet'/>
<propval name='mtu' type='count' value='9000'/>
</property_group>
<property_group name='net2' type='datalink-phys'>
<propval name='devname' type='astring' value='i40e2'/>
<propval name='loc' type='astring' value='/SYS/MB'/>
<propval name='media' type='astring' value='Ethernet'/>
</property_group>
<property_group name='net3' type='datalink-phys'>
<propval name='devname' type='astring' value='i40e3'/>
<propval name='loc' type='astring' value='/SYS/MB'/>
<propval name='media' type='astring' value='Ethernet'/>
</property_group>
<property_group name='sp-phys0' type='datalink-phys'>
<propval name='devname' type='astring' value='usbecm2'/>
<propval name='media' type='astring' value='Ethernet'/>
</property_group>
</property_group>
<property_group name='linkname-policy' type='application'>
<propval name='initialized' type='boolean' value='true'/>
</property_group>
</instance>
</service>
</service_bundle>
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='profile' name='extract'>
<service name='network/ip-interface-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='interfaces' type='application'>
<property_group name='aggr0' type='interface-ip'>
<property name='address-family' type='astring'>
<astring_list>
<value_node value='ipv4'/>
<value_node value='ipv6'/>
</astring_list>
</property>
<property_group name='v4' type='address-static'>
<propval name='ipv4-address' type='astring' value='198.51.100.63'/>
<propval name='prefixlen' type='count' value='24'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
<property_group name='v6' type='address-addrconf'>
<propval name='interface-id' type='astring' value='::'/>
<propval name='prefixlen' type='count' value='0'/>
<propval name='stateful' type='astring' value='no'/>
<propval name='stateless' type='astring' value='no'/>
</property_group>
<property_group name='v6a' type='address-static'>
<propval name='ipv6-address' type='astring' value='2001:db8:414:60bc::3f'/>
<propval name='prefixlen' type='count' value='64'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
</property_group>
<property_group name='lo0' type='interface-loopback'>
<property name='address-family' type='astring'>
<astring_list>
<value_node value='ipv4'/>
<value_node value='ipv6'/>
</astring_list>
</property>
<property_group name='v4' type='address-static'>
<propval name='ipv4-address' type='astring' value='127.0.0.1'/>
<propval name='prefixlen' type='count' value='8'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
<property_group name='v6' type='address-static'>
<propval name='ipv6-address' type='astring' value='::1'/>
<propval name='prefixlen' type='count' value='128'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
</property_group>
<property_group name='sp-phys0' type='interface-ip'>
<property name='address-family' type='astring'>
<astring_list>
<value_node value='ipv4'/>
<value_node value='ipv6'/>
</astring_list>
</property>
<property_group name='v4' type='address-static'>
<propval name='ipv4-address' type='astring' value='203.0.113.77'/>
<propval name='prefixlen' type='count' value='24'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
</property_group>
</property_group>
<property_group name='ipmgmtd' type='application'>
<propval name='datastore_version' type='integer' value='2047'/>
</property_group>
<property_group name='static-routes' type='application'>
<property_group name='route-1' type='static-route'>
<propval name='destination' type='astring' value='default'/>
<propval name='family' type='astring' value='inet'/>
<propval name='gateway' type='astring' value='198.51.100.1'/>
</property_group>
</property_group>
</instance>
</service>
</service_bundle>
Step 4: Reinstall staging system with full network configuration
The combined contents of base-profile.xml from Step 1 and labnet-profile.xml from step 3 could be used as the SC-profile for autoconfiguring headbanger on a fresh install. However, my goal is to easily replicate this configuration to many systems. I made replication easier by moving system-specific details from these two files to a smaller system-specific SC-profile file. This is all just cutting and pasting existing XML text blocks.
headbanger-profile.xml
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='profile' name='headbanger-profile'>
<service name='system/identity' type='service' version='0'>
<instance name='node' enabled='true'>
<property_group name='config' type='application'>
<propval name='nodename' type='astring' value='headbanger'/>
</property_group>
</instance>
</service>
<service name='network/datalink-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='datalinks' type='application'>
<property_group name='aggr0' type='datalink-aggr'>
<propval name='probe-ip' type='astring' value='198.51.100.63+198.51.100.1'/>
</property_group>
</property_group>
</instance>
</service>
<service name='network/ip-interface-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='interfaces' type='application'>
<property_group name='aggr0' type='interface-ip'>
<property_group name='v4' type='address-static'>
<propval name='ipv4-address' type='astring' value='198.51.100.63'/>
<propval name='prefixlen' type='count' value='24'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
<property_group name='v6a' type='address-static'>
<propval name='ipv6-address' type='astring' value='2001:db8:414:60bc::3f'/>
<propval name='prefixlen' type='count' value='64'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
</property_group>
</property_group>
<property_group name='static-routes' type='application'>
<property_group name='route-1' type='static-route'>
<propval name='destination' type='astring' value='default'/>
<propval name='family' type='astring' value='inet'/>
<propval name='gateway' type='astring' value='198.51.100.1'/>
</property_group>
</property_group>
</instance>
</service>
</service_bundle>
The two original SC-profile files now contain:
base-profile.xml
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type="profile" name="sysconfig">
<service version="1" type="service" name="system/config-user">
<instance enabled="true" name="default">
<property_group type="application" name="root_account">
<propval type="astring" name="login" value="root"/>
<propval type="astring" name="password" value="$5$rounds=10000$HJbjRmpw$JehxKTyBZxAbZmq0HN7gi367n8b4tcs1GkdgzaRbjk6"/>
<propval type="astring" name="type" value="normal"/>
</property_group>
</instance>
</service>
<service version="1" type="service" name="system/timezone">
<instance enabled="true" name="default">
<property_group type="application" name="timezone">
<propval type="astring" name="localtime" value="US/Eastern"/>
</property_group>
</instance>
</service>
<service version="1" type="service" name="system/environment">
<instance enabled="true" name="init">
<property_group type="application" name="environment">
<propval type="astring" name="LANG" value="C"/>
</property_group>
</instance>
</service>
<service version="1" type="service" name="system/keymap">
<instance enabled="true" name="default">
<property_group type="system" name="keymap">
<propval type="astring" name="layout" value="US-English"/>
</property_group>
</instance>
</service>
<service version="1" type="service" name="system/console-login">
<instance enabled="true" name="default">
<property_group type="application" name="ttymon">
<propval type="astring" name="terminal_type" value="sun-color"/>
</property_group>
</instance>
</service>
</service_bundle>
labnet-profile.xml
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='profile' name='labnet-profile'>
<service name='network/datalink-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='datalinks' type='application'>
<property_group name='aggr0' type='datalink-aggr'>
<propval name='aggr-mode' type='astring' value='dlmp'/>
<propval name='force' type='boolean' value='false'/>
<propval name='key' type='count' value='0'/>
<propval name='lacp-mode' type='astring' value='off'/>
<propval name='lacp-timer' type='astring' value='short'/>
<propval name='media' type='astring' value='Ethernet'/>
<propval name='num-ports' type='count' value='2'/>
<propval name='policy' type='astring' value='L4'/>
<property name='ports' type='astring'>
<astring_list>
<value_node value='net0'/>
<value_node value='net1'/>
</astring_list>
</property>
</property_group>
<property_group name='net0' type='datalink-phys'>
<propval name='mtu' type='count' value='9000'/>
</property_group>
<property_group name='net1' type='datalink-phys'>
<propval name='mtu' type='count' value='9000'/>
</property_group>
</property_group>
</instance>
</service>
<service name='network/ip-interface-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='interfaces' type='application'>
<property_group name='aggr0' type='interface-ip'>
<property name='address-family' type='astring'>
<astring_list>
<value_node value='ipv4'/>
<value_node value='ipv6'/>
</astring_list>
</property>
<property_group name='v6' type='address-addrconf'>
<propval name='interface-id' type='astring' value='::'/>
<propval name='prefixlen' type='count' value='0'/>
<propval name='stateful' type='astring' value='no'/>
<propval name='stateless' type='astring' value='no'/>
</property_group>
</property_group>
</property_group>
</instance>
</service>
</service_bundle>
Do the following on the AI server.
Copy the three profile files to the AI server then use installadm(8) then apply them to the original nettest-sparc configuration. I found installadm's ability to use multiple SC-profiles on a single service with client selection criteria is extremely handy for this situation.
# installadm update-profile -n nettest-sparc \
-p base-profile \
-f base-profile.xml
# installadm create-profile -n nettest-sparc \
-p labnet-profile \
-f labnet-profile.xml
# installadm create-profile -n nettest-sparc \
-p headbanger-profile \
-f headbanger-profile.xml \
-c mac=0:0:5e:0:53:d2
Do the following on the staging system.
Log into the staging system's console and reinstall.
root@headbanger:~# halt
{0} ok boot net:dhcp - install
This repeats the installation of the staging system as before, but with the full network configuration automatically applied on first boot. After installation is complete, log in and verify the system is configured as intended.
Step 5. Repeat installation on other systems
Duplicating the network configuration on other systems is relatively easy now. In this example, I will recreate the configuration on node "beatnik".
Do the following on the AI server.
Start by copying the headbanger-profile.xml from step 4 to beatnik-profile.xml then use an editor to provide beatnik's unique configuration details. Bold fonts show where I made changes.
beatnik-profile.xml
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='profile' name='beatnik-profile'>
<service name='system/identity' type='service' version='0'>
<instance name='node' enabled='true'>
<property_group name='config' type='application'>
<propval name='nodename' type='astring' value='beatnik'/>
</property_group>
</instance>
</service>
<service name='network/datalink-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='datalinks' type='application'>
<property_group name='aggr0' type='datalink-aggr'>
<propval name='probe-ip' type='astring' value='198.51.100.64+198.51.100.1'/>
</property_group>
</property_group>
</instance>
</service>
<service name='network/ip-interface-management' type='service' version='0'>
<instance name='default' enabled='true'>
<property_group name='interfaces' type='application'>
<property_group name='aggr0' type='interface-ip'>
<property_group name='v4' type='address-static'>
<propval name='ipv4-address' type='astring' value='198.51.100.64'/>
<propval name='prefixlen' type='count' value='24'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
<property_group name='v6a' type='address-static'>
<propval name='ipv6-address' type='astring' value='2001:db8:414:60bc::40'/>
<propval name='prefixlen' type='count' value='64'/>
<propval name='up' type='astring' value='yes'/>
</property_group>
</property_group>
</property_group>
<property_group name='static-routes' type='application'>
<property_group name='route-1' type='static-route'>
<propval name='destination' type='astring' value='default'/>
<propval name='family' type='astring' value='inet'/>
<propval name='gateway' type='astring' value='198.51.100.1'/>
</property_group>
</property_group>
</instance>
</service>
</service_bundle>
Use installadm(8) on the install server to load beatnik's installation details.
# installadm create-client -n nettest-sparc \
-e 0:0:5e:0:53:24
# installadm create-profile -n nettest-sparc \
-p beatnik-profile \
-f beatnik-profile.xml \
-c mac=0:0:5e:0:53:24
The following installadm list command shows how selection criteria is used to identify which SC-profile files are applied during the installation of each system.
# installadm list -p -n nettest-sparc
Service Name Profile Name Environment Criteria
------------ ------------ ----------- --------
nettest-sparc base-profile install none
system
labnet-profile install none
system
headbanger-profile install mac=0:0:5e:0:53:d2
system
beatnik-profile install mac=0:0:5e:0:53:24
system
Do the following on other systems targeted for the same configuration.
Log into the console and start the installation.
{0} ok boot net:dhcp - install
The full network configuration will automatically be applied on first boot of beatnik just as it was on headbanger.