Thursday Oct 08, 2009

IPv6 in Shared-Stack Zones

I was recently at an OpenSolaris user-group meeting where a question was asked regarding how IPv6 could be used from a shared-stack zone.  For the benefit of anyone who has a similar question, here is an example of a working configuration:

bash-3.2# zoneadm list -iv
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              native   shared
   - test             installed  /export/home/test              native   excl  
   - test2            installed  /export/home/test2             native   shared

The exclusive-stack zone "test" has all of its own networking configured within it, so IPv6 inherently just works there.  The question, however, was about shared-stack, and so I setup the "test2" zone to demonstrate this.

bash-3.2# zonecfg -z test2
zonecfg:test2> add net
zonecfg:test2:net> set physical=e1000g0
zonecfg:test2:net> set address=fe80::1234/10
zonecfg:test2:net> end
zonecfg:test2> add net
zonecfg:test2:net> set physical=e1000g0
zonecfg:test2:net> set address=2002:a08:39f0:1::1234/64
zonecfg:test2:net> end
zonecfg:test2> verify
zonecfg:test2> commit
zonecfg:test2> exit
bash-3.2# zonecfg -z test2 info
zonename: test2
zonepath: /export/home/test2
brand: native
...
net:
	address: 10.8.57.111/24
	physical: e1000g0
	defrouter not specified
net:
	address: fe80::1234/10
	physical: e1000g0
	defrouter not specified
net:
	address: 2002:a08:39f0:1::1234/64
	physical: e1000g0
	defrouter not specified

Here I configured a link-local address fe80::1234/10, and a global address 2002:a08:39f0:1::1234/64.  Each interface within each zone requires a link-local address for use with neighbor-discovery, and the global address is the address used for actual IPv6 communication by applications and services. The global address' prefix is one that is configured on the link to which the interface is connected.  In the zone, we end up with:

bash-3.2# zlogin test2 ifconfig -a6
lo0:1: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
	inet6 ::1/128 
e1000g0:2: flags=2000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2
	inet6 fe80::1234/10 
e1000g0:3: flags=2000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2
	inet6 2002:a08:39f0:1::1234/64 

The global zone has IPv6 connectivity using this same prefix as well as a default IPv6 route: [2]

bash-3.2# netstat -f inet6 -rn

Routing Table: IPv6
  Destination/Mask            Gateway                   Flags Ref   Use    If   
--------------------------- --------------------------- ----- --- ------- ----- 
2002:a08:39f0:1::/64        2002:a08:39f0:1:214:4fff:fe1e:1e72 U       1       0 e1000g0:1 
fe80::/10                   fe80::214:4fff:fe1e:1e72    U       1       0 e1000g0 
default                     fe80::1                     UG      1       0 e1000g0 
::1                         ::1                         UH      1      21 lo0   

From the non-global zone, we have IPv6 connectivity:

bash-3.2# zlogin test2 ping -sn 2002:8194:aeaa:1:214:4fff:fe70:5530
PING 2002:8194:aeaa:1:214:4fff:fe70:5530 (2002:8194:aeaa:1:214:4fff:fe70:5530): 56 data bytes
64 bytes from 2002:8194:aeaa:1:214:4fff:fe70:5530: icmp_seq=0. time=4.654 ms
64 bytes from 2002:8194:aeaa:1:214:4fff:fe70:5530: icmp_seq=1. time=2.632 ms
64 bytes from 2002:8194:aeaa:1:214:4fff:fe70:5530: icmp_seq=2. time=2.501 ms
64 bytes from 2002:8194:aeaa:1:214:4fff:fe70:5530: icmp_seq=3. time=2.571 ms
\^C
----2002:8194:aeaa:1:214:4fff:fe70:5530 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms)  min/avg/max/stddev = 2.501/3.090/4.654/1.044

The zone can then be configured to use DNS or local hosts to resolve names to IPv6 addresses in order to utilize IPv6 more effectively.

Tuesday Nov 18, 2008

Observe Loopback and Inter-Zone IP Packets With OpenSolaris

I'm happy to announce that the IP Observability Devices component of the Clearview project has integrated into OpenSolaris build 103 (also see Phil Kirk's announcement to the ON community).  This adds the following new capabilities to OpenSolaris:

  • Network observability at the IP layer for traditional DLPI-based tools such as snoop
  • Observability of loopback IP packets
  • Observability of inter-zone IP packets
  • Tools such as snoop can be run from within a non-global zone to observe packets associated with that zone
  • Snoop filtering based on zone id

The snoop command has grown a new "-I <interface-name>" option to access this feature.  Its semantics are to snoop the IP interface named <interface-name> at the IP layer.  When observing a particular IP interface with this facility, packets that have a source or destination IP address assigned to that interface can be observed, as well as packets that are forwarded to or from that IP interface, and broadcast and multicast packets received by that interface.  Additional internal filtering is performed to ensure that an observer from a non-global zone can only see packets that belong to that zone, with the exception of the global zone, from which packets to or from any zone that shares its stack can be observed.  Any IP interface visible through "ifconfig -a" can be observed using this feature.

We are also working towards integrating support for these IP Observability Devices into Wireshark and tcpdump in the near future.

Here are some examples using snoop:

Example 1: Observing the Loopback Interface

bash-3.2# snoop -I lo0
Using device ipnet/lo0 (promiscuous mode)
   localhost -> localhost    ICMP Echo request (ID: 37110 Sequence number: 0)
   localhost -> localhost    ICMP Echo reply (ID: 37110 Sequence number: 0)

The lo0 interface has the 127.0.0.1 address assigned to it, and so any communication using the address 127.0.0.1 is seen above (in this case, I was simply doing "ping 127.0.0.1").  Snoop's verbose output mode displays a new "ipnet" header that precedes all IP packets observed:

bash-3.2# snoop -v -I lo0
Using device ipnet/lo0 (promiscuous mode)
IPNET:  ----- IPNET Header -----
IPNET:  
IPNET:  Packet 1 arrived at 10:40:33.68506
IPNET:  Packet size = 108 bytes
IPNET:  dli_version = 1
IPNET:  dli_type = 4
IPNET:  dli_srczone = 0
IPNET:  dli_dstzone = 0
IPNET:  
IP:   ----- IP Header -----
IP:   
IP:   Version = 4
IP:   Header length = 20 bytes
...

Note above that the source and destination zone ids are displayed.  In this case, I was running "ping 127.0.0.1" in the global zone, and so both the source and destination zone ids are "0".

Example 2: Running Snoop From a Non-Global Zone

bash-3.2# zoneadm list -v
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              native   shared
   4 test             running    /zones/test                    native   shared
bash-3.2# zlogin test
[Connected to zone 'test' pts/2]
...
bash-3.2# ifconfig -a
lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000 
bge0:1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
	inet 10.8.57.34 netmask ffffff00 broadcast 10.8.57.255
lo0:1: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128 
bge0:2: flags=202000841<UP,RUNNING,MULTICAST,IPv6,CoS> mtu 1500 index 2
        inet6 2002:a08:39f0:1::f/64 
bash-3.2# snoop -I bge0
Using device ipnet/bge0 (promiscuous mode)
whitestar1-2.East.Sun.COM -> mf-ubur-01.East.Sun.COM DNS C 253.57.8.10.in-addr.arpa. Internet PTR ?
mf-ubur-01.East.Sun.COM -> whitestar1-2.East.Sun.COM DNS R 2.0.0.224.in-addr.arpa. Internet PTR ALL-ROUTERS.MCAST.NET.
whitestar1-6.East.Sun.COM -> whitestar1-2.East.Sun.COM TCP D=22 S=62117 Syn Seq=195630514 Len=0 Win=49152 Options=<mss
whitestar1-2.East.Sun.COM -> whitestar1-6.East.Sun.COM TCP D=62117 S=22 Syn Ack=195630515 Seq=195794440 Len=0 Win=49152
whitestar1-6.East.Sun.COM -> whitestar1-2.East.Sun.COM TCP D=22 S=62117 Ack=195794441 Seq=195630515 Len=0 Win=49152
whitestar1-2.East.Sun.COM -> whitestar1-6.East.Sun.COM TCP D=62117 S=22 Push Ack=195630515 Seq=195794441 Len=20 Win=491

Although not evident from the snoop output above, whitestar1-2 is 10.8.57.34 (the bge0:1 IP address in this non-global zone), and whitestar1-6 is actually an IP address in another zone on the same system.  By snooping the bge0 interface, the user sees all packets associated with the bge0 IP addresses in the zone; even those that are locally delivered to other zones.  Using snoop's verbose output mode allows us to see which zones these packets are flowing between:

bash-3.2# snoop -v -I bge0 whitestar1-6
Using device ipnet/bge0 (promiscuous mode)
IPNET:  ----- IPNET Header -----
IPNET:  
IPNET:  Packet 1 arrived at 10:44:10.86739
IPNET:  Packet size = 76 bytes
IPNET:  dli_version = 1
IPNET:  dli_type = 4
IPNET:  dli_srczone = 0
IPNET:  dli_dstzone = 4
IPNET:  
IP:   ----- IP Header -----
IP:   
IP:   Version = 4
IP:   Header length = 20 bytes
...

We can see above that the packet was from the global zone to the test zone.

Example 3: Filtering by Zone ID

Filtering by zone id can be useful on a system that has multiple zones.  In this example, an administrator in the global zone observes packets being sent to or from IP addresses in the "test" zone.

bash-3.2# zoneadm list -v
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              native   shared
   4 test             running    /zones/test                    native   shared
bash-3.2# snoop -I bge0 zone 4
Using device ipnet/bge0 (promiscuous mode)
whitestar1-6.East.Sun.COM -> whitestar1-2.East.Sun.COM TCP D=22 S=61658 Syn Seq=374055417 Len=0 Win=49152 Options=<mss
whitestar1-2.East.Sun.COM -> whitestar1-6.East.Sun.COM TCP D=61658 S=22 Syn Ack=374055418 Seq=374124525 Len=0 Win=49152
whitestar1-6.East.Sun.COM -> whitestar1-2.East.Sun.COM TCP D=22 S=61658 Ack=374124526 Seq=374055418 Len=0 Win=49152

This can be particularly useful with the loopback interface, as the 127.0.0.1 address is shared among all shared-stack zones, and it can be difficult to associate a loopback packet to an application in a zone.

Note that there is a pending RFE to also be able to enter a zone name as well as a zone id as the argument to the snoop "zone" filtering primitive.  For now, the zone id is the only allowable argument.

About

user12618912

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
News
Blogroll

No bookmarks in folder