DSCP IP Addresses
By Bob Hueston on Jan 16, 2008
DSCP IP Address ReuseOne customer emailed me to ask if it was OK to use the same DSCP IP address range on multiple machines in the same datacenter. The short answer is yes.
The only requirement for the DSCP addresses is that they not be used elsewhere on the external networks, either the network used by the SCF or by the Solaris domains. Clearly if you use the same IP addresses for some machines in the datacenter as are used for either the SCF or the Solaris domains, then the SCF and Solaris domains would not be able to connect to those other machines; requests to those IP addresses would be routed to the internal DSCP network, not to the external Ethernet ports.
It is perfectly acceptable to use the same DSCP addresses on every SPARC Enterprise machine in the datacenter. In our development lab, we had well over a dozen machines on a single subnet (both the SCF and the Solaris domains), and they all shared a common set of DSCP IP addresses. We operated that way for over a year without a problem.
We could have hard-coded the DSCP addresses, but it always seems whatever address you select, at least one customer is using that same address in their datacenter. So after consulting with manufacturing and Sun Service, we decided to leave the DSCP IP addresses entirely up to the customer.
DSCP Network Address and NetmaskAnother customer raised an interesting question. They had a SPARC Enterprise machine which is capable of supporting multiple Solaris domains, but they were only configuring it into a single domain. When they tried to use
setdscpon the SCF, they provided a network address and netmask that only contained two IP addresses, one for the SCF and one for the Solaris domain.
setdscpdoesn't allow that; it insists that the netmask be large enough to handle the maximum number of domains that the chassis can support. This is so that
setdscpcan compute IP addresses for the SCF and all possible Solaris domains.
But this customer was adamant that they could only spare two IP addresses for DSCP.
setdscp can be invoked in three ways. The first method:
is the one recommended by the user documentation. This takes an IP network address and a netmask and computed IP addresses for the SCF and all the possible domains. This method is provided purely as a convenience. The network address and netmask are not used at all by DSCP, other than to compute individual IP addresses.
setdscp -i address -m netmask
You can also invoke
setdscp with no arguments, and it will prompt you for a network address and netmask and compute IP addresses for the SCF and all domains. In effect, this is the same as the first method, except it's interactive.
The third method of using
setdscp is to manually assign the SCF and domain DSCP IP addresses one at a time using the format:
The first line sets the SCF's DSCP interface to a specific IP address. The second line sets each domain's DSCP interface IP address. You can invoke the second line once for each domain you plan on configuring.
setdscp -s -i address setdscp -d domain_id -i address
Caveat: With the second approach, you can configure just some of the domains you plan on creating. For example, you may have an M9000, capable of 24 domains, but you only plan on using it as a single system, so you configure the SCF and domain 0's DSCP IP addresses. Later, though, you may decide to create a new domain 1. You can add boards to domain 1, but when you try to power it on, the
poweron command will fail, because there's no DSCP address for domain 1. Last time I checked (this was in XCP1040) the failure occurred late in the boot process, after the
poweron command had returned to the user saying the domain was being powered on. As a result, the only way to see why the domain didn't power on was to inspect the error logs. (Note: This behavior may have changed in later versions of the SCF firmware.)