My thoughts on configuring zones with shared IP instances and the 'defrouter' parameter

An occasional call or email I receive has questions about routing issues when using Solaris Zones in the (default) shared IP Instance configuration. Everything works well when the non-global zones are on the same IP subnet (lets say 172.16.1.0/24) as the global zone. Routing gets a little tricky when the non-global zones are on a different subnet.

My general recommendation is to isolate. This means:

  • Separate subnets for the global zone (administration, backup) and the non-global zones (applications, data).
  • Separate data-links for the global and non-global zones.
    • The non-global zones can share a data-link
    • Non-global zones on different IP subnets use different data-links
Using separate data-links is not always possible. I was concerned whether this would actually work.

So I did some testing, and exchanged some emails because of a comment I made regarding PSARC/2008/057 and the automatic removal of a default route when the zone is halted.

Turns out I have been very restrictive in suggesting that the global and non-global zones not share a data-link. While I think that is a good administrative policy, to separate administrative and application traffic, it is not a requirement. It is OK to have the global zone and one or more non-global zones share the same data-link. However, if the non-global zones are to have different default routes, they must be on subnets that the global zone is not on.

My test case running Solaris 10 10/09 has the global zone on the 129.154.53.0/24 network and the non-global zone on the 172.16.27.0/24 network.

global# ifconfig -a
...
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 129.154.53.132 netmask ffffff00 broadcast 129.154.53.255
        ether 0:14:4f:ac:57:c4
e1000g0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        zone shared1
        inet 172.16.27.27 netmask ffffff00 broadcast 172.16.27.255

global# zonecfg -z shared1 info net
net:
        address: 172.16.27.27/24
        physical: e1000g0
        defrouter: 172.16.27.16

The routing table as seen from both are:
global# netstat -rn

Routing Table: IPv4
  Destination           Gateway           Flags  Ref     Use     Interface
-------------------- -------------------- ----- ----- ---------- ---------
default              129.154.53.215       UG        1        123
default              172.16.27.16         UG        1          7 e1000g0
129.154.53.0         129.154.53.132       U         1         50 e1000g0
224.0.0.0            129.154.53.132       U         1          0 e1000g0
127.0.0.1            127.0.0.1            UH        3         80 lo0

shared1# netstat -rn

Routing Table: IPv4
  Destination           Gateway           Flags  Ref     Use     Interface
-------------------- -------------------- ----- ----- ---------- ---------
default              172.16.27.16         UG        1          7 e1000g0
172.16.27.0          172.16.27.27         U         1          3 e1000g0:1
224.0.0.0            172.16.27.27         U         1          0 e1000g0:1
127.0.0.1            127.0.0.1            UH        4         78 lo0:1
While the global zone shows both routes, only the default applying to its subnet will be used. And for traffic leaving the non-global zone, only its default will be used.

You may notice that the Interface for the global zone's default router is blank. That is because I have set the default route via /etc/defaultrouter. I noticed that if it is determined via the route discovery daemon, it will be listed as being on e1000g0! This does not affect the behavior, however it may be visually confusing, which is probably why I initially leaned towards saying to not share the data-link.

There are multiple ways to determining which route might be used, including ping(1M) and traceroute(1M). I like the output of the route get command.

global# route get 172.16.29.1
   route to: 172.16.29.1
destination: default
       mask: default
    gateway: 129.154.53.1
  interface: e1000g0
      flags: <UP,GATEWAY,DONE,STATIC>
 recvpipe  sendpipe  ssthresh    rtt,ms rttvar,ms  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0

shared1# route get 172.16.28.1
   route to: 172.16.28.1
destination: default
       mask: default
    gateway: 172.16.27.16
  interface: e1000g0:1
      flags: <UP,GATEWAY,DONE,STATIC>
 recvpipe  sendpipe  ssthresh    rtt,ms rttvar,ms  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0
This quickly shows which interfaces and IP addresses are being used. If there are multiple default routes, repeated invocations of this will show a rotation in the selection of the default routes.

Thanks to Erik Nordmark and Penny Cotten for their insights on this topic!

Steffen Weiberle

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

stw

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today