DSCP IP Addresses

On the subject of Sun SPARC Enterprise M-Class server Domain-to-SCF Communication Protocol (I wrote about it yesterday), I received two questions about the IP addresses reserved for the internal DSCP network. The first has to do with reusing the IP addresses in multiple machines; the second has to do with the DSCP netmask.

DSCP IP Address Reuse

One customer emailed me to ask if it was OK to use the same DSCP IP address range on multiple machines in the same datacenter. The short answer is yes.

The only requirement for the DSCP addresses is that they not be used elsewhere on the external networks, either the network used by the SCF or by the Solaris domains. Clearly if you use the same IP addresses for some machines in the datacenter as are used for either the SCF or the Solaris domains, then the SCF and Solaris domains would not be able to connect to those other machines; requests to those IP addresses would be routed to the internal DSCP network, not to the external Ethernet ports.

It is perfectly acceptable to use the same DSCP addresses on every SPARC Enterprise machine in the datacenter. In our development lab, we had well over a dozen machines on a single subnet (both the SCF and the Solaris domains), and they all shared a common set of DSCP IP addresses. We operated that way for over a year without a problem.

We could have hard-coded the DSCP addresses, but it always seems whatever address you select, at least one customer is using that same address in their datacenter. So after consulting with manufacturing and Sun Service, we decided to leave the DSCP IP addresses entirely up to the customer.

DSCP Network Address and Netmask

Another customer raised an interesting question. They had a SPARC Enterprise machine which is capable of supporting multiple Solaris domains, but they were only configuring it into a single domain. When they tried to use setdscp on the SCF, they provided a network address and netmask that only contained two IP addresses, one for the SCF and one for the Solaris domain. setdscp doesn't allow that; it insists that the netmask be large enough to handle the maximum number of domains that the chassis can support. This is so that setdscp can compute IP addresses for the SCF and all possible Solaris domains.

But this customer was adamant that they could only spare two IP addresses for DSCP.

In actuality, setdscp can be invoked in three ways. The first method:

        setdscp -i address -m netmask
is the one recommended by the user documentation. This takes an IP network address and a netmask and computed IP addresses for the SCF and all the possible domains. This method is provided purely as a convenience. The network address and netmask are not used at all by DSCP, other than to compute individual IP addresses.

You can also invoke setdscp with no arguments, and it will prompt you for a network address and netmask and compute IP addresses for the SCF and all domains. In effect, this is the same as the first method, except it's interactive.

The third method of using setdscp is to manually assign the SCF and domain DSCP IP addresses one at a time using the format:

        setdscp -s -i address
        setdscp -d domain_id -i address
The first line sets the SCF's DSCP interface to a specific IP address. The second line sets each domain's DSCP interface IP address. You can invoke the second line once for each domain you plan on configuring.

Caveat: With the second approach, you can configure just some of the domains you plan on creating. For example, you may have an M9000, capable of 24 domains, but you only plan on using it as a single system, so you configure the SCF and domain 0's DSCP IP addresses. Later, though, you may decide to create a new domain 1. You can add boards to domain 1, but when you try to power it on, the poweron command will fail, because there's no DSCP address for domain 1. Last time I checked (this was in XCP1040) the failure occurred late in the boot process, after the poweron command had returned to the user saying the domain was being powered on. As a result, the only way to see why the domain didn't power on was to inspect the error logs. (Note: This behavior may have changed in later versions of the SCF firmware.)


In summary, you can use the same DSCP IP addresses for all Sun SPARC Enterprise M-Class machines in your datacenter. Given that, it should be rare that you run low on IP addresses for DSCP, but if you do, know that you can manually configure only the DSCP IP addresses you really need.

This is a great summary of DSCP. Should be in an Infodoc on Sunsolve. Had a cu ask questions regarding DSCP today and this by far was the best resource.

David M.

Posted by David McDivitt on May 11, 2009 at 08:17 AM EDT #

Post a Comment:
Comments are closed for this entry.

Bob Hueston


Top Tags
« July 2016