Using Solaris and SPARC Networking and Virtualization

Using IP Instances with VLANs or How to Make a Few NICs Look Like Many

[Minor editorial and clarification updates 2009.09.28]

Solaris 10 8/07 includes a new feature for zone networking. IP Instances is the facility to give a non-global zone its own complete control over the IP stack, which previously was shared with and controlled by the global zone.

A zone that has an exclusive IP Instance can set interface parameters using ifconfig(1M), put an interface into promiscuous mode to run snoop(1M), be a DHCP client or server, set ndd(1M) variables, have its own IPsec policies, etc.

One requirement for an exclusive IP Instance is that it must have exclusive access to a link name. This is any NIC, VLAN-tagged NIC component, or aggregation at this time. When they become available, virtual NICs will make this much simpler, as a single NIC can be presented to the zones using a number of VNICs, effectively multiplexing access to that NIC. A link name is an entry that can be found in /dev, such as /dev/bge0, /dev/bge321001 (VLAN tag 321 on bge1), aggr2, and so on.

To see what link names are available on a system, use dladm(1M) with the show-link option. For example:

global# dladm show-link
bge0 type: non-vlan mtu: 1500 device: bge0
bge1 type: non-vlan mtu: 1500 device: bge1
bge2 type: non-vlan mtu: 1500 device: bge2
bge3 type: non-vlan mtu: 1500 device: bge3

As folks have started to use IP Instances to isolate their zones, they have noticed that they don't have sufficient link names (I'll use just link in the rest of this) to assigned to the zones that have or wish to configure as exclusive. So, how does a global zone administrator configure a large number of zones as exclusive?

Let's consider the following situation, where there are three tiers of a web service, where each tier is on a different network.

If each server has only one NIC, the total number of switch ports required is at least eight (8). If each server has a management port, that is another eight ports, even if they are on a different, management network. Add to that at least three three switch ports going to the router.

Consolidating the servers onto a single Solaris 10 instance using exclusive IP Instances requires at least eight NICs for the services (one per service), and at least one for the global zone and management. (We'll ignore a service process requirements, since they are separate anyway, and access could be either via a serial interface or a network.)

One option to consider is using VLANs and VLAN tagging. When using VLAN tagging, additional information is put onto the ethernet frame by the sender which allows the receiver to associated that frame to a specific VLAN. The specification allow up to 4094 VLAN tags, from 1 to 4094. For more information on administering VLANs in Solaris 10, see Administering Virtual Local Area Networks in the Solaris 10 System Administrator Collection.

VLANs is a method to collapse multiple ethernet broadcast domains (whether hubs or switches) into a single network unit (usually a switch). [Typically, a single IP subnet, such as, is on a broadcast domain. Within such a switch frame, you can have a large number of virtual switches, consolidating network infrastructure and still isolating broadcast domains. Often, the use of VLANs is completely hidden from the systems tied to the switch, as a port on the switch is configured for only one VLAN. With VLAN tagging, a single port can allow a system to connect to multiple VLAns, and therefore multiple networks. Both the switch and the system must be configured for VLAN tagging for this to work properly. VLAN tagging has been used for years, and is robust and reliable.

Any one network interface can have multiple VLANs configured for it, but a single VLAN ID can only exist once on each interface. Thus it is possible to put multiple networks or broadcast domains on a single interface. It is not possible to put more than one VLAN of any broadcast domain on a single interface. For example, you can put VLANs 111, 112, and 113 on interface bge1, but you can not put VLAN 111 on bge1 more than once. You can, however, put VLAN 111 on interfaces bge1 and bge2.

Using the case shown above, if the three web servers are on the same network, say, you would want to have three interfaces that are all connected to a VLAN capable switch, and configure each interface with a VLAN tag that is the same as the VLAN ID on the switch.

For example, if the VLAN tag is 111 and the interfaces are bge1 through bge3, the link names you would assign to the three web servers would be bge111001, bge111002, and bge111003.

Introducing zones into the setup, the web servers can be run in three separate zones, and with exclusive IP Instances, they can be totally separate and each assigned a VLAN-tagged interface. Web Server 1 could have bge111001, Web Server 2 could have bge111002, and Web Server 3 could have bge111003.

global# zonecfg -z web1 info net
address not specified
physical: bge111001
global# zonecfg -z web2 info net
address not specified
physical: bge111002
global# zonecfg -z web3 info net
address not specified
physical: bge111003

Within the zones, you could configure IP addresses through

Similarly, for the authentication tier, using VLAN ID 112, you could assign the zones auth1 through auth3 to bge112001, bge112002, and bge112003,respectively. And for application servers app1 and app2 on VLAN ID 113, bge113001 and bge113002. This can be repeated until some limit is reached, whether it is network bandwidth, system resource limits, or the maximum number of concurrent VLANs on either the switch or Solaris.

This configuration could look like the following diagram.

Web Server 1, Auth Server 1, and Application Server 1 share the use of NIC1, yet are all on different VLANs (111, 112, and 113, respectively). The same for instances 2 and 3, except that there is no third application server. All traffic between the three web servers will stay within the switch, as will traffic between the authentication servers. Traffic between the tiers is passed between the IP networks by the router. NICg is showing that the global zone also has a network interface.

Using this technique, the maximum number of zones with exclusive IP Instances you could deploy on a single system that are on the same subnet is limited to the number of interfaces that are capable of doing VLAN tagging. In the above example, with three bge interfaces on the system, the maximum number of exclusive zones on a single subnet would be three. (I have intentionally reserved bge0 for the global zone, but it would be possible to use it as well, making sure the global zone uses a different VLAN ID altogether, such as 1 or 2.)

Join the discussion

Comments ( 6 )
  • nyohadi Friday, November 16, 2007

    Dear Sir ,

    Your article very good to help me configure vlan on solaris 10 u4(0807) .

    I am configure on SF240 and Global zone use bge0(ip

    Also i am configure 2 x bigzone /full zone and assign each zone ip exclusive and 2 vlan(vlan 111 and 222) in bge0 interface .

    bash-3.00# zoneadm list -cv


    0 global running / native shared

    2 splunk running /data/splunk native shared

    8 zone1 running /zones/zone1 native excl

    9 zone2 running /zones/zone2 native excl

    bash-3.00# dladm show-link

    bge0 type: non-vlan mtu: 1500 device: bge0

    bge111000 type: vlan 111 mtu: 1500 device: bge0

    bge222000 type: vlan 222 mtu: 1500 device: bge0

    bge1 type: non-vlan mtu: 1500 device: bge1

    bge2 type: non-vlan mtu: 1500 device: bge2

    bge3 type: non-vlan mtu: 1500 device: bge3

    i set ip bge111000 is

    bge222000 is

    My Question :

    1. I cannot ping ip or from global zone or outside network.

    2. How to configure vlan 111 and vlan 222 ? so zone1 and zone2 can communicate with outside network ?



  • steffen Friday, November 16, 2007

    Hi Hadi,

    Each VLAN is like a separate switch. VLAN 111 is on one switch, VLAN 222 is on a second switch, and the non-VLAN (bge0) is on a third switch. These switches just happen to be contained in a single piece of hardware and are completely configurable via software.

    Therefore, unless you connect the different switches together, systems on any one switch can not communicate with those on any of the other switches. I don't know if there are switches out there that allow you to bridge between VLANs, essentially cascading switches and putting multiple VLANs on the same broadcast domain.

    You have several choices. For all zones on network, use a single VLAN, for example 111, and all four interfaces (bge111000, bge111001, bge111002, and bge111003). Or put them on different subnets, and use a router between them (which is the example I used in the article). Or physically wire the VLANs together; but that is not desirable.

    You can not easily use VLANs to make the system look like it has more interfaces on a single switch and broadcast domain.

    Hope this helps.


  • Nicolas Dorfsman Thursday, December 6, 2007


    For french-speaking visitors, I wrote an article on our french wiki about configuring VLAN in Solaris :


  • Al Sunday, August 29, 2010

    Hello Steffen,

    It's possible to create 2 datalink names / VLANs on a given NIC and use one of them by the global zone while the other by some exclusive IP zone. The NIC in this case carries traffic with 2 different VLAN IDs.

    My question is, can we do the above but have the traffic from the global zone un-tagged? If yes, can you please indicate what datalink name needs to be plumbed in the global zone in such case.



  • Steffen Weiberle Monday, August 30, 2010

    Hi Al,

    It is possible to use any combination of untagged (e.g. bge0, e1000g3) and tagged (bge111000 and e1000g111003) when using zones, whether shared or exclusive. (In any decision, to make VLANs easier to understand, just replace the VLAN tagged links with non-tagged. For example, replace bge111000 with mylink1, and e1000g111003 with mylink2, and make your decisions based on that.)

    So given the above example links, you could have bge0 in your global zone, and use e1000g3, bge111000, and e1000g111003 for your zones. Or you could have e1000g111003 in your global zone, and use bge0, e1000g3, and bge111000 for your zones.

    Remember that there can only be on specific tag, lets say 111, per physical link (there is only one bge111000). And anything exclusive is dedicated to the zone that is currently running and has it assigned.


    PS. If you have more questions, it might good to provide your specific information, and to send that via email to my @ oracle . com address. Name hasn't changed as its a unique combination.

  • niman Friday, September 17, 2010


    When I try to configure zone with "set ip-type=exclusive" it shows usage. Solaris version is 10 11/06? What is the issue?

Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.