Brussels and data link administration

Solaris has a lot of cutting-edge tunable features implemented in its network drivers, but administering these features remains a somewhat chaotic story.Even a frequently tuned property like Jumbo Frame MTU can be quite a bear to administer, as Shantnu recently discovered.

 And there are more of these instances. For example, driver writers are frequently confused about the expected semantics for ethernet flow-control in solaris . Even Solaris engineers are sometimes confounded by the existing definitions.

 The good news is that Sun is working on improving these interfaces and bringing in a cleaner administrative interface. The project is Brussels and its objectives are to pull together the best of the exisiting methods, so that

  • like ndd, we can configure properties on the fly,
  • like driver.conf, we can configure properties like Jumbo Frames (but without the teeth gnashing syntax) and have them persist across reboot
  • leverage from the flexible syntax  and other features (like the show- subcommands) introduced in dladm(1m), while making this  play  nicely with smf
  • have a uniform, intuitive syntax for configuring properties (no more head scratching over whether its link_duplex or link_mode, and if it should be 0 for half-duplex, or 1 for half-duplex!)

The most important requirement to make this succesful is feedback from Administrators about what you would like to see implemented in Brussels- so please contribute your input! Here are some examples of the improvements being introduced by Brussels..

 For example, Brussels introduces the "show-ether" sub-command in dladm that will allow the administrator to view the status of the ethernet network. In the vanilla invocation,

# dladm show-ether bge1
LINK          PTYPE    STATE    AUTO SPEED-DUPLEX    PAUSE
bge1          current  up       yes  1G-f             tx


Which says that the bge1 interface is UP (i.e., the driver is is RUNNING)  with autonegotiation enabled, speed 1Gbps, full duplex. It also shows that flow control is in the "tx" (Transmit) direction only, i.e., we will send pause frames when congested, but ignore any received pause frames from the peer.

I can get "extended" output :

# dladm show-ether -x bge1
LINK     PTYPE    STATE    AUTO SPEED-DUPLEX    PAUSE
bge1     current  up       yes  1G-f              tx
--       capable  --       yes  1G-fh,100M-fh,10M-fh      bi
--       adv     --       yes  1G-f                      tx
--       peeradv  --       yes  1G-f                      bi

 
The additional rows tell me the hardware capabilities of the local endpoint ("capable") , those advertised to the peer ("adv") and those advertised by the peer ("peeradv").Note that the speed-duplex capabilites of bge1 are (1G, full duplex), (1G, half-duplex), (1000 Mbps, full duplex), (1000 Mbps, half duplex), (10 Mbps, full duplex), (10Mbps, half duplex). Although my bge1 driver is capable of bi-directional flow-control, it has been administratively configured to advertise TX only. Since the peer is advertising bi-directional flow control,  I  try:

 

# dladm set-linkprop -p flowctrl=bad bge1    # I have a syntax error!
dladm: warning: link property 'flowctrl' must be one of: no,rx,tx,bi

So I correct my error, and now have the state:

# dladm set-linkprop -p flowctrl=bi bge1  
# dladm show-ether -x bge1               
LINK    PTYPE    STATE    AUTO SPEED-DUPLEX          PAUSE
bge1    current  up       yes  1G-f                  both
--      capable  --       yes  1G-fh,100M-fh,10M-fh  both
--      adv     --       yes  1G-f                  both
--      peeradv  --       yes  1G-f                  both


We are also adding better observability into the kernel state- see Artem's blog on the new mdb macros that will make it possible to look at a crash dump or a running kernel's mdb state and figure out what customizations (including Private properties, known only to the driver!) have been applied.

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

sowmini

Search

Categories
Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today