I am showing this with Solaris 11.3, though the steps will work for any Solaris 11. Due to networking differences in Solaris 10, the principles will apply yet the steps will be different.
I find it useful for my understanding to see not just command line input. I also like to see the output, and validation that steps I perform actually do something. When doing network configurations, I prefer to see network traffic. This session will include all of that. For networking that requires a second system, and I will show the setup of that as well. I hope this is useful for others.
Initial network configuration is as follows. It has some other things on it, that I am cutting from the output as it is not relevant to this topic.
root@remote# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet up 1000 full e1000g1
net2 Ethernet up 1000 full e1000g2
net0 Ethernet up 1000 full e1000g0
net3 Ethernet up 1000 full e1000g3
root@remote# dladm show-link
LINK CLASS MTU STATE OVER
net1 phys 1500 up --
net2 phys 1500 up --
net0 phys 1500 up --
net3 phys 9000 up --
root@remote# dladm show-vlan
root@remote#
root@remote# dladm create-vlan -l net3 -v 111 net3111
root@remote# dladm create-vlan -l net3 -v 112 net3111
root@remote# dladm create-vlan -l net3 -v 113 net3113
root@remote#
root@remote# dladm show-link
LINK CLASS MTU STATE OVER
net1 phys 1500 up --
net2 phys 1500 up --
net0 phys 1500 up --
net3 phys 9000 up --
net3111 vlan 9000 up net3
net3112 vlan 9000 up net3
net3113 vlan 9000 up net3
root@remote#
root@remote# dladm show-vlan
LINK VID SVID PVLAN-TYPE FLAGS OVER
net3111 111 -- -- ----- net3
net3112 112 -- -- ----- net3
net3113 113 -- -- ----- net3
root@remote#
Next step is to put some IP address on those VLAN. I use 192.168.VLAN.x as my subnet, and I set "x" to the host part of the IP address of the base system, in this case "1".
root@remote# ipadm create-ip net3111
root@remote# ipadm create-ip net3112
root@remote# ipadm create-ip net3113
root@remote#
root@remote# ipadm create-addr -a 192.168.111.1/24 net3111
net3111/v4
root@remote# ipadm create-addr -a 192.168.112.1/24 net3112
net3112/v4
root@remote# ipadm create-addr -a 192.168.113.1/24 net3113
net3113/v4
root@remote#
root@remote# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 172.16.1.1/22
net3111/v4 static ok 192.168.111.1/24
net3112/v4 static ok 192.168.112.1/24
net3113/v4 static ok 192.168.113.1/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::214:4fff:feac:57c4/10
net0/v6 addrconf ok 2606:b400:602:c080:214:4fff:feac:57c4/64
root@remote#
root@cdom# dladm create-aggr -l net1 -l net3 aggr1
root@cdom#
root@cdom# dladm show-aggr
LINK MODE POLICY ADDRPOLICY LACPACTIVITY LACPTIMER
aggr1 trunk L4 auto off short
root@cdom#
root@cdom# dladm show-aggr -P
LINK MODE POLICY ADDRPOLICY LACPACTIVITY LACPTIMER
aggr1 trunk L4 auto off short
root@cdom# dladm show-aggr -L
LINK PORT AGGREGATABLE SYNC COLL DIST DEFAULTED EXPIRED
aggr1 net1 no no no no no no
-- net3 no no no no no no
root@cdom# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr1 -- 1000Mb full up 0:21:28:d2:17:f9 --
net1 1000Mb full up 0:21:28:d2:17:f9 attached
net3 1000Mb full up 0:21:28:d2:17:fb attached
root@cdom#
The aggregation is on a private network on the Netgear switch, so a snoop will not show a lot of traffic. I will generate some traffic using ping, and I will be switching between the two systems for that.
root@remote# ping 192.168.111.101 2
no answer from 192.168.111.101
root@remote# ping 192.168.112.101 2
no answer from 192.168.112.101
root@remote# ping 192.168.113.101 2
no answer from 192.168.113.101
root@remote#
root@cdom# snoop -d aggr1
Using device aggr1 (promiscuous mode)
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.101, 192.168.111.101 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.101, 192.168.111.101 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.1, 192.168.111.1 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.101, 192.168.111.101 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.1, 192.168.112.1 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.101, 192.168.112.101 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.101, 192.168.112.101 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.101, 192.168.112.101 ?
VLAN#113: 192.168.113.1 -> (broadcast) ARP C Who is 192.168.113.1, 192.168.113.1 ?
VLAN#113: 192.168.113.1 -> (broadcast) ARP C Who is 192.168.113.101, 192.168.113.101 ?
VLAN#113: 192.168.113.1 -> (broadcast) ARP C Who is 192.168.113.101, 192.168.113.101 ?
VLAN#113: 192.168.113.1 -> (broadcast) ARP C Who is 192.168.113.101, 192.168.113.101 ?
You can see the ARP requests for all three VLANs with the target address on each. This is why I like to have the VLAN ID and the subnet the same. I am beginning to notice this with some customers as well.
My first test is to bring down one or both ports and see the changes in the aggregation and the network. After turning off the port of net1, this is how things look.
root@cdom# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr1 -- 1000Mb full up 0:21:28:d2:17:f9 --
net1 0Mb unknown down 0:21:28:d2:17:f9 standby
net3 1000Mb full up 0:21:28:d2:17:fb attached
root@cdom#
root@cdom# snoop -d aggr1
Using device aggr1 (promiscuous mode)
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.101, 192.168.111.101 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.101, 192.168.111.101 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.101, 192.168.111.101 ?
^Croot@cdom#
root@cdom# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr1 -- 0Mb unknown down 0:21:28:d2:17:f9 --
net1 0Mb unknown down 0:21:28:d2:17:f9 standby
net3 0Mb unknown down 0:21:28:d2:17:fb standby
root@cdom#
root@cdom# tail /var/adm/messages
...
Apr 21 11:00:07 gravity mac: [ID 486395 kern.info] NOTICE: igb3 link down
Apr 21 11:05:40 gravity mac: [ID 486395 kern.info] NOTICE: igb1 link down
Apr 21 11:05:40 gravity mac: [ID 486395 kern.info] NOTICE: aggr1 link down
root@cdom#
root@cdom# dladm create-vlan -l aggr1 -v 111 aggr1111
root@cdom# ipadm create-ip aggr1111
root@cdom# ipadm create-addr -a 192.168.111.5/24 aggr1111
aggr1111/v4
root@cdom#
root@cdom# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.1.5/22
net4/v4 static ok 169.254.182.77/24
aggr1111/v4 static ok 192.168.111.5/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::221:28ff:fed2:17f8/10
net0/v6 addrconf ok 2606:b400:602:c080:221:28ff:fed2:17f8/64
root@cdom#
root@remote# ping 192.168.111.5 2
192.168.111.5 is alive
root@remote#
root@cdom# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr1 -- 1000Mb full up 0:21:28:d2:17:f9 --
net1 1000Mb full up 0:21:28:d2:17:f9 attached
net3 1000Mb full up 0:21:28:d2:17:fb attached
root@cdom#
root@cdom# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr1 -- 1000Mb full up 0:21:28:d2:17:f9 --
net1 0Mb unknown down 0:21:28:d2:17:f9 standby
net3 1000Mb full up 0:21:28:d2:17:fb attached
root@cdom#
root@remote# ping 192.168.111.5 2
192.168.111.5 is alive
root@remote#
root@cdom# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr1 -- 0Mb unknown down 0:21:28:d2:17:f9 --
net1 0Mb unknown down 0:21:28:d2:17:f9 standby
net3 0Mb unknown down 0:21:28:d2:17:fb standby
root@cdom#
root@cdom# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.1.5/22
net4/v4 static ok 169.254.182.77/24
aggr1111/v4 static inaccessible 192.168.111.5/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::221:28ff:fed2:17f8/10
net0/v6 addrconf ok 2606:b400:602:c080:221:28ff:fed2:17f8/64
root@cdom#
root@remote# ping 192.168.111.5 2
no answer from 192.168.111.5
root@remote#
This is an area where there are differences between Solaris 11 and Solaris 10 Service Domains. Oracle highly recommends that all Service Domains are running Solaris 11.
I will be creating a virtual switch on top of the aggr1 data link while the existing VLANs are already there. If this were Solaris 10, I'd likely remove them and if I need Service Domain access to the aggregation, I would use the virtual switch.
In Solaris 11, there is no need to set or modify the pvid and vid parameters on the virtual switch. If this were Solaris 10 and I wanted to get access to VLANs on the data link (in this case aggr1) I would need to set those.
Let us get started on the virtual switch.
root@cdom# ldm add-vsw net-dev=aggr1 primary-vsw1 primary
root@cdom#
root@cdom# ldm list-services
...
VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE INTER-VNET-LINK
primary-vsw0 primary 00:14:4f:f9:b4:9f net0 0 switch@0 1 1 1500 on
primary-vsw1 primary 00:14:4f:f8:44:87 aggr1 1 switch@1 1 1 1500 on
...
root@cdom#
root@cdom# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 16 7680M 0.2% 0.2% 69d 17h 38m
host1 active -n---- 5000 8 4G 0.2% 0.2% 5d 21h 23m
root@cdom#
guest@host1:~$ pfbash [1]
guest@host1:~$
guest@host1:~$ PS1="guest-pf@host1$ "
guest-pf@host1$
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
guest-pf@host1$
guest-pf@host1$ dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
guest-pf@host1$
root@cdom# ldm add-vnet linkprop=phys-state vnet1 primary-vsw1 host1
root@cdom#
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
net1 Ethernet unknown 0 unknown vnet1
guest-pf@host1$
[1] You may wonder what I did here. If you don't, skip to the next section.
I am using Solaris' Role Based Access Control feature, where this user has been given a lot of privileges. I could just do an su(1) to root. Instead, I am running as the user in the profile shell version of bash. Every command is then checked for authorization. This is easier than running "pfexec command", for those familiar with sudo, "sudo command". The pfexec(1) command does not prompt for a password.
root@cdom# ldm set-vnet vid=111 vnet1 host1
root@cdom#
guest-pf@host1$ snoop -d net1
Using device net1 (promiscuous mode)
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
^Cguest-pf@host1$
root@cdom# ldm set-vnet vid=111,112 pvid=113 vnet1 host1
root@cdom#
guest-pf@host1$ snoop -d net1
Using device net1 (promiscuous mode)
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
192.168.113.1 -> (broadcast) ARP C Who is 192.168.113.6, 192.168.113.6 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
192.168.113.1 -> (broadcast) ARP C Who is 192.168.113.6, 192.168.113.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
192.168.113.1 -> (broadcast) ARP C Who is 192.168.113.6, 192.168.113.6 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
^Cguest-pf@host1$
Note that I can add or remove VLANs while the interface is running in the Guest Domain.
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
net1 Ethernet down 0 unknown vnet1
guest-pf@host1$
guest-pf@host1$ ipadm create-ip net1
guest-pf@host1$ ipadm create-addr -a 192.168.113.6/24 net1
net1/v4
guest-pf@host1$
guest-pf@host1$ ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.1.6/22
net1/v4 static ok 192.168.113.6/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::214:4fff:fef9:fc75/10
net0/v6 addrconf ok 2606:b400:602:c080:214:4fff:fef9:fc75/64
guest-pf@host1$
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
net1 Ethernet up 0 unknown vnet1
guest-pf@host1$
guest-pf@host1$ snoop -d net1
Using device net1 (promiscuous mode)
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
192.168.113.1 -> 192.168.113.6 ICMP Echo request (ID: 26169 Sequence number: 8405)
192.168.113.6 -> 192.168.113.1 ICMP Echo reply (ID: 26169 Sequence number: 8405)
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
VLAN#111: 192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
192.168.113.1 -> 192.168.113.6 ICMP Echo request (ID: 26169 Sequence number: 8406)
192.168.113.6 -> 192.168.113.1 ICMP Echo reply (ID: 26169 Sequence number: 8406)
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
^Cguest-pf@host1$
guest-pf@host1$
I quickly tested the configuration with one port on the switch down, and in the guest it looks the same. Then I marked the second port down. This makes the aggregation down.
root@cdom# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr1 -- 0Mb unknown down 0:21:28:d2:17:f9 --
net1 0Mb unknown down 0:21:28:d2:17:f9 standby
net3 0Mb unknown down 0:21:28:d2:17:fb standby
root@cdom#
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
net1 Ethernet down 0 unknown vnet1
guest-pf@host1$
guest-pf@host1$ ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.1.6/22
net1/v4 static inaccessible 192.168.113.6/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::214:4fff:fef9:fc75/10
net0/v6 addrconf ok 2606:b400:602:c080:214:4fff:fef9:fc75/64
guest-pf@host1$
guest-pf@host1$ snoop -d net1
Using device net1 (promiscuous mode)
^Cguest-pf@host1$
guest-pf@host1$
To demonstrate, I will turn linkprop off, and then look at the interface in the Guest Domain.
root@cdom# ldm set-vnet linkprop="" vnet1 host1
root@cdom#
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
net1 Ethernet up 0 unknown vnet1
guest-pf@host1$
guest-pf@host1$ ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.1.6/22
net1/v4 static ok 192.168.113.6/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::214:4fff:fef9:fc75/10
net0/v6 addrconf ok 2606:b400:602:c080:214:4fff:fef9:fc75/64
guest-pf@host1$
Let's give it a try.
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
net1 Ethernet up 0 unknown vnet1
guest-pf@host1$
guest-pf@host1$ dladm show-phys -m
LINK SLOT ADDRESS INUSE CLIENT
net0 primary 0:14:4f:f9:fc:75 yes net0
1 0:14:4f:fb:a1:78 no --
2 0:14:4f:f8:f9:32 no --
3 0:14:4f:f9:ab:37 no --
4 0:14:4f:f8:1:93 no --
net1 primary 0:14:4f:f8:3e:e5 yes net1
guest-pf@host1$
guest-pf@host1$ dladm create-vnic -l net1 vnic11
dladm: vnic creation failed: operation not supported
guest-pf@host1$
guest-pf@host1$ dladm create-vnic -l net0 vnic1
guest-pf@host1$
guest-pf@host1$ dladm show-phys -m
LINK SLOT ADDRESS INUSE CLIENT
net0 primary 0:14:4f:f9:fc:75 yes net0
1 0:14:4f:fb:a1:78 yes vnic1
2 0:14:4f:f8:f9:32 no --
3 0:14:4f:f9:ab:37 no --
4 0:14:4f:f8:1:93 no --
net1 primary 0:14:4f:f8:3e:e5 yes net1
guest-pf@host1$
This is also why I chose to show VNICs outside of Zones. If we had gone straight to Zone creating and start-up, this failure might be harder to track down.
Several years ago LDoms added a new feature to assign additional MAC addresses to a vnet. The property is called "alt-mac-addrs". It allows a fixed number of MAC addresses to be assigned to the vnet. Unfortunately, this vnet property can not be set or changed when a Guest Domain is running. So I will shut the Guest down.
guest-pf@host1$ init 5
updating /platform/sun4v/boot_archive
guest-pf@host1$
root@cdom# ldm set-vnet alt-mac-addrs=auto,auto,auto,auto,auto,auto vnet1 host1
Please perform the operation while the LDom is bound or inactive
root@cdom#
root@cdom# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 16 7680M 1.3% 1.3% 69d 22h 15m
host1 active -n---- 5000 8 4G 0.1% 0.1% 3h 54m
root@cdom#
root@cdom# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 16 7680M 0.5% 0.5% 69d 22h 16m
host1 bound ------ 5000 8 4G
root@cdom#
root@cdom# ldm set-vnet alt-mac-addrs=auto,auto,auto,auto,auto,auto vnet1 host1
root@cdom#
root@cdom# ldm start host1
LDom host1 started
root@cdom#
Once the Guest Domain is back up, and can see what things look like now.
guest-pf@host1$ dladm show-phys -m
LINK SLOT ADDRESS INUSE CLIENT
net0 primary 0:14:4f:f9:fc:75 yes net0
1 0:14:4f:fb:a1:78 yes vnic1
2 0:14:4f:f8:f9:32 no --
3 0:14:4f:f9:ab:37 no --
4 0:14:4f:f8:1:93 no --
net1 primary 0:14:4f:f8:3e:e5 yes net1
1 0:14:4f:fa:a6:5e no --
2 0:14:4f:f8:92:c0 no --
3 0:14:4f:f9:77:8c no --
4 0:14:4f:fb:d8:33 no --
5 0:14:4f:f8:50:1 no --
6 0:14:4f:fa:bc:2d no --
guest-pf@host1$
guest-pf@host1$ dladm create-vnic -l net1 vnic11
guest-pf@host1$
guest-pf@host1$ dladm show-phys -m
LINK SLOT ADDRESS INUSE CLIENT
net0 primary 0:14:4f:f9:fc:75 yes net0
1 0:14:4f:fb:a1:78 yes vnic1
2 0:14:4f:f8:f9:32 no --
3 0:14:4f:f9:ab:37 no --
4 0:14:4f:f8:1:93 no --
net1 primary 0:14:4f:f8:3e:e5 yes net1
1 0:14:4f:fa:a6:5e yes vnic11
2 0:14:4f:f8:92:c0 no --
3 0:14:4f:f9:77:8c no --
4 0:14:4f:fb:d8:33 no --
5 0:14:4f:f8:50:1 no --
6 0:14:4f:fa:bc:2d no --
guest-pf@host1$
guest-pf@host1$ dladm show-vnic
LINK OVER SPEED MACADDRESS MACADDRTYPE IDS
vnic1 net0 0 0:14:4f:fb:a1:78 factory, slot 1 VID:0
vnic11 net1 0 0:14:4f:fa:a6:5e factory, slot 1 VID:0
guest-pf@host1$
guest-pf@host1$ dladm delete-vnic vnic1
guest-pf@host1$
guest-pf@host1$ dladm show-vnic
LINK OVER SPEED MACADDRESS MACADDRTYPE IDS
vnic11 net1 0 0:14:4f:fa:a6:5e factory, slot 1 VID:0
guest-pf@host1$
guest-pf@host1$ dladm create-vnic -l net1 -v 111 vnic1111
guest-pf@host1$
guest-pf@host1$ dladm show-vnic
LINK OVER SPEED MACADDRESS MACADDRTYPE IDS
vnic11 net1 0 0:14:4f:fa:a6:5e factory, slot 1 VID:0
vnic1111 net1 0 0:14:4f:f8:92:c0 factory, slot 2 VID:111
guest-pf@host1$ dladm show-vlan
guest-pf@host1$
guest-pf@host1$ ipadm create-ip vnic1111
guest-pf@host1$ ipadm create-addr -a 192.168.111.6/24 vnic1111
vnic1111/v4
guest-pf@host1$
guest-pf@host1$ snoop -d net1
Using device net1 (promiscuous mode)
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
VLAN#111: 192.168.111.1 -> 192.168.111.6 ICMP Echo request (ID: 26167 Sequence number: 13612)
VLAN#111: 192.168.111.6 -> 192.168.111.1 ICMP Echo reply (ID: 26167 Sequence number: 13612)
192.168.113.1 -> 192.168.113.6 ICMP Echo request (ID: 26169 Sequence number: 13601)
192.168.113.6 -> 192.168.113.1 ICMP Echo reply (ID: 26169 Sequence number: 13601)
VLAN#111: 192.168.111.6 -> (broadcast) ARP C Who is 192.168.111.6, 192.168.111.6 ?
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
VLAN#111: 192.168.111.1 -> 192.168.111.6 ICMP Echo request (ID: 26167 Sequence number: 13613)
VLAN#111: 192.168.111.6 -> 192.168.111.1 ICMP Echo reply (ID: 26167 Sequence number: 13613)
192.168.113.1 -> 192.168.113.6 ICMP Echo request (ID: 26169 Sequence number: 13602)
192.168.113.6 -> 192.168.113.1 ICMP Echo reply (ID: 26169 Sequence number: 13602)
VLAN#112: 192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
^Cguest-pf@host1$
And snoop now shows that pings are working on 192.168.113.6 and now 192.168.111.6. Now I will disable both interfaces on the switch.
guest-pf@host1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 0 unknown vnet0
net1 Ethernet down 0 unknown vnet1
guest-pf@host1$
guest-pf@host1$ dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
vnic11 vnic 1500 up net1
vnic1111 vnic 1500 down net1
guest-pf@host1$
guest-pf@host1$ ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.1.6/22
net1/v4 static ok 192.168.113.6/24
vnic1111/v4 static inaccessible 192.168.111.6/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::214:4fff:fef9:fc75/10
net0/v6 addrconf ok 2606:b400:602:c080:214:4fff:fef9:fc75/64
guest-pf@host1$
guest-pf@host1$ snoop -d net1
Using device net1 (promiscuous mode)
^Cguest-pf@host1$
Note: I may come back to this later and update details.
Let us move on to Zones.
I will not focus on the mechanics of creating a Solaris Zone here. Others and I have done that elsewhere. However, the network details of the Zone configuration are important to highlight.
guest-pf@host1$ zonecfg -z myzone info anet
anet:
linkname: net0
lower-link: net1
allowed-address not specified
configure-allowed-address: true
defrouter not specified
allowed-dhcp-cids not specified
link-protection: mac-nospoof
mac-address: auto
mac-prefix not specified
mac-slot not specified
vlan-id not specified
priority not specified
rxrings not specified
txrings not specified
mtu not specified
maxbw not specified
bwshare not specified
rxfanout not specified
vsi-typeid not specified
vsi-vers not specified
vsi-mgrid not specified
etsbw-lcl not specified
cos not specified
pkey not specified
linkmode not specified
evs not specified
vport not specified
anet:
linkname: net1
lower-link: net1
...
vlan-id: 111
...
anet:
linkname: net2
lower-link: net1
...
vlan-id: 112
...
guest-pf@host1$
The link "net0" had the defaults, and is using the net1 interface. Since "vlan-id" is not specified, it will use the untagged inteface, or VLAN 113.
The other two interfaces, net1 and net2 will use VLAN IDs 111 and 112, respectively.
Because I did not give guest all Zone privileges, I perform a few operations here as root. User guest can start and stop Zones, and also log into the Zone.
guest-pf@host1$ su
Password:
root@host1:~#
root@host1:~# zonecfg -z myzone -f myzone.cfg
UX: /usr/sbin/usermod: guest is currently logged in, some changes may not take effect until next login.
root@host1:~#
root@host1:~# zoneadm -z myzone install -c myzone.xml
The following ZFS file system(s) have been created:
rpool/zones
rpool/zones/myzone
Progress being logged to /var/log/zones/zoneadm.20160421T225323Z.myzone.install
Image: Preparing at /zones/myzone/root.
Install Log: /system/volatile/install.1585/install_log
AI Manifest: /tmp/manifest.xml.P6aOed
SC Profile: /export/home/guest/myzone.xml
Zonename: myzone
Installation: Starting ...
Creating IPS image
Startup linked: 1/1 done
Installing packages from:
solaris
origin: http://172.16.1.1/
DOWNLOAD PKGS FILES XFER (MB) SPEED
Completed 279/279 48306/48306 354.4/354.4 1.6M/s
PHASE ITEMS
Installing new actions 66017/66017
Updating package state database Done
Updating package cache 0/0
Updating image state Done
Creating fast lookup database Done
Updating package cache 1/1
Installation: Succeeded
Note: Man pages can be obtained by installing pkg:/system/manual
done.
Done: Installation completed in 431.151 seconds.
Next Steps: Boot the zone, then log into the zone console (zlogin -C)
to complete the configuration process.
Log saved in non-global zone as /zones/myzone/root/var/log/zones/zoneadm.20160421T225323Z.myzone.install
root@host1:~#
root@host1:~# exit
exit
guest-pf@host1$
guest-pf@host1$ zoneadm -z myzone boot
guest-pf@host1$
guest-pf@host1$ zlogin myzone
[Connected to zone 'myzone' pts/2]
Last login: Thu Apr 21 19:15:02 2016 on pts/2
Oracle CorporationSunOS 5.1111.3February 2016
root@myzone:~#
root@myzone:~# dladm show-phys
root@myzone:~# dladm show-link
LINK CLASS MTU STATE OVER
net2 vnic 1500 up ?
net1 vnic 1500 up ?
net0 vnic 1500 up ?
root@myzone:~#
root@myzone:~# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.113.16/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::214:4fff:fef8:5001/10
root@myzone:~#
root@myzone:~# ping 192.168.113.1
192.168.113.1 is alive
root@myzone:~#
root@myzone:~# snoop -d net0
Using device net0 (promiscuous mode)
^Croot@myzone:~#
root@myzone:~#
root@myzone:~# snoop -d net1
Using device net1 (promiscuous mode)
192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.2, 192.168.111.2 ?
192.168.111.1 -> (broadcast) ARP C Who is 192.168.111.2, 192.168.111.2 ?
^Croot@myzone:~#
root@myzone:~# snoop -d net2
Using device net2 (promiscuous mode)
192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
192.168.112.1 -> (broadcast) ARP C Who is 192.168.112.6, 192.168.112.6 ?
^Croot@myzone:~#
I think the final item I want to show is the link failure as seen in the Zone.
root@myzone:~# dladm show-link
LINK CLASS MTU STATE OVER
net2 vnic 1500 down ?
net1 vnic 1500 up ?
net0 vnic 1500 up ?
guest-pf@host1$ dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
vnic11 vnic 1500 up net1
vnic1111 vnic 1500 up net1
myzone/net2 vnic 1500 down net1
myzone/net1 vnic 1500 up net1
myzone/net0 vnic 1500 up net1
guest-pf@host1$
I hope it is useful for you!
Regards,
Steffen
Thanks to Jeff Savit for a quick review and editorial suggestions. He and I discussed the need this topic several times.
(Other than minor typographical changes)
2016.04.22: Posted
2016.04.21: Created