Flow control in Solaris 11 Express Network virtualization
By jsavit on Jan 31, 2011
Flow control in Solaris 11 Express
One of the powerful new features of Solaris 11 Express is Network Virtualization, and in this blog I will illustrate just one of its many capabilities. Rather than describe Solaris 11 networking enhancements, I recommend you read Jeff Victor's blog entry on this topic at http://blogs.oracle.com/JeffV/category/Solaris+11". Go there, have a nice read and then come back here. :-)
Now that you're back, consider an environment with multiple networked hosts such as in figure 1, with hosts on different switches and with differing bandwidth requirements.
To consolidate them, we may need to preserve the network topology for isolation purposes, and ensure that the consolidated systems each have guaranteed bandwidth to meet their service level objectives. For example, we may want to prevent file transfer or backup traffic from slowing down interactive response traffic for transactional systems.
In Solaris 10 we can easily consolidate multiple servers onto multiple zones in a single Solaris instance. This eliminates the need for physical networks for the consolidated applications to communicate with one another, but preserving the network topology and managing quality of service was not as straightforward. Now, in Solaris 11 Express, we can describe virtual networks to maintain isolation and different network properties, and we can establish flow settings to manage quality of service. It is possible to create virtual network configurations that match the physical one, and control performance properties, as shown in figure 2:
Let's illustrate the quality of service aspect by defining flows on physical network devices. We could just as easily do so on virtual network objects, but I want to start with familiar things.
In the example below, I create a flow named "flow1" that is set to a maximum of 2Mb per second. This flow is associated with the network device (also called a "link") e1000g1, is temporary (the "-t") and doesn't persist over a system boot, and is in effect only for a specific remote host. Perhaps that's the IP address of a user who hasn't been very nice to me, so I want to slow down his network access to this server. :-) The first line defines the flow, and the second line sets its maximum bandwidth.
Additionally, a second flow called "http1" is set on network device e1000g0 that limits access to the web server (listening on port 80) to only 8Mb per second.
# flowadm add-flow -t -l e1000g1 -a remote_ip=192.168.56.5 flow1 # flowadm set-flowprop -p maxbw=2M flow1 # flowadm add-flow -t -l e1000g0 -a transport=tcp,local_port=80 -p maxbw=8M http-1 # flowadm show-flow FLOW LINK IPADDR PROTO PORT DSFLD http-1 e1000g0 -- tcp 80 -- flow1 e1000g1 RMT:192.168.56.5/32 -- -- -- # flowadm show-flowprop FLOW PROPERTY VALUE DEFAULT POSSIBLE http-1 maxbw 8 -- http-1 priority -- -- flow1 maxbw 2 -- 2M flow1 priority -- --
To illustrate the first flow rule, I created a file called "lotsa_junk",
and transmitted it via
scp from the host with IP address 192.168.56.5.
Before adding flow1, it took about 10 seconds:
$ scp lotsa_junk 192.168.56.101: lotsa_junk 100% |**********************************| 97 MB 00:11
After adding flow
$ scp lotsa_junk 192.168.56.101: lotsa_junk 100% |**********************************| 97 MB 03:57
This is kind of fun to watch on the screen (if you are easily amused, that is) since the first example quickly draws asterisks across the screen to show progress, while the second one slooooowwwwlllly indicates progress.
I usually get bored with this while it's running - the point has been made after just a few seconds - and issue a "flowadm set-flowprop" to increase the bandwidth. That illustrates a second point: flow properties can be dynamically changed as needed. In the following snippet I use "flowstat" to watch the traffic rate while playing with it, and then increase the bandwidth limit.
# flowstat -i 10 flow1 FLOW IPKTS RBYTES IERRS OPKTS OBYTES OERRS flow1 7.45K 11.16M 0 16.58K 1.22M 0 flow1 1.10K 1.66M 0 765 57.65K 0 flow1 1.10K 1.66M 0 765 57.70K 0 flow1 1.10K 1.66M 0 769 58.03K 0 [[[ this is too painful. speed it up! ]]] # flowadm set-flowprop -p maxbw=9M flow1 # flowstat -i 10 flow1 FLOW IPKTS RBYTES IERRS OPKTS OBYTES OERRS flow1 24.04K 36.18M 0 28.07K 2.08M 0 flow1 6.97K 10.45M 0 3.88K 270.61K 0 flow1 7.06K 10.61M 0 3.94K 273.89K 0 flow1 6.89K 10.34M 0 3.88K 271.36K 0