Solaris 11 Express Network Tunables

Overview

For years I, and many others, have been tuning TCP, UDP, IP, and other aspects of the Solaris network stack with ndd(1M). The ndd command is documented, however, most of the tunables were really private interface implementations, subject to change, and lacked documentation in many cases. Also, ndd does not show the default values, nor the possible values or ranges..

That is changing with Solaris 11 Express. A new command ipadm(1M) allows persistent and temporary (with the -t option) setting of key tunable values. This is a major improvement over ndd, where it is customary to create an /etc/rc2.d/S69ndd or similar script to set the parameter on every reboot. Another benefit is that ipadm shows the default value and the values that the property can be set to.

The ipadm has many features to configure the IP settings of interfaces. This blog entry focuses on how ipadm replaces ndd. Note that ipadm only supports the IP, TCP, UDP, SCTP, and ICMP protocols. Other protocols such as ipsecah and keysock still required the use of ndd.

Review of ndd

To get a list of all tunables for a specific protocol, an ndd -get operation is performed with "?" as the argument. For example, this is a way of listing all the TCP parameters.
root@Solaris11Express# ndd -get /dev/tcp \?
tcp_time_wait_interval         (read and write)
tcp_conn_req_max_q             (read and write)
tcp_conn_req_max_q0            (read and write)
tcp_conn_req_min               (read and write)
...
tcp_dev_flow_ctl               (read and write)
tcp_reass_timeout              (read and write)
tcp_extra_priv_ports_add       (write only)
tcp_extra_priv_ports_del       (write only)
tcp_extra_priv_ports           (read only)
tcp_1948_phrase                (write only)
tcp_listener_limit_conf        (read only)
tcp_listener_limit_conf_add    (write only)
tcp_listener_limit_conf_del    (write only)
To get the current value of specific parameter, list the parameter as the argument for the driver, in this case /dev/tcp.
root@Solaris11Express# ndd -get /dev/tcp tcp_conn_req_max_q
128
And to set parameter, follow it with a value.
root@Solaris11Express# ndd -set /dev/tcp tcp_conn_req_max_q 256
root@Solaris11Express# ndd -get /dev/tcp tcp_conn_req_max_q
256
And for my own benefit, I set it back to the original.
root@Solaris11Express# ndd -set /dev/tcp tcp_conn_req_max_q 128
root@Solaris11Express# ndd -get /dev/tcp tcp_conn_req_max_q
128

Using the ipadm *-prop Options

The ipadm(1M) manual page lists three sub-commands to manage TCP/IP protocol properties.
     ipadm set-prop [-t] -p prop=value[,...] protocol
     ipadm reset-prop [-t] -p prop protocol
     ipadm show-prop [[-c] -o field[,...]] [-p prop[,...]] [protocol]
To list all the properties for all the protocols as currently supported, I run ipadm with the show-prop sub-command.
root@Solaris11Express# ipadm show-prop
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
ipv4  forwarding            rw   off          --           off          on,off
ipv4  ttl                   rw   255          --           255          1-255
ipv6  forwarding            rw   off          --           off          on,off
ipv6  hoplimit              rw   255          --           255          1-255
ipv6  hostmodel             rw   weak         --           weak         strong,
                                                                        src-priority,
                                                                        weak
ipv4  hostmodel             rw   weak         --           weak         strong,
                                                                        src-priority,
                                                                        weak
icmp  recv_maxbuf           rw   8192         --           8192         4096-65536
icmp  send_maxbuf           rw   8192         --           8192         4096-65536
tcp   ecn                   rw   passive      --           passive      never,passive,
                                                                        active
tcp   extra_priv_ports      rw   2049,4045    --           2049,4045    1-65535
tcp   largest_anon_port     rw   65535        --           65535        1024-65535
tcp   recv_maxbuf           rw   128000       --           128000       2048-1073741824
tcp   sack                  rw   active       --           active       never,passive,
                                                                        active
tcp   send_maxbuf           rw   49152        --           49152        4096-1073741824
tcp   smallest_anon_port    rw   32768        --           32768        1024-65535
tcp   smallest_nonpriv_port rw   1024         --           1024         1024-32768
udp   extra_priv_ports      rw   2049,4045    --           2049,4045    1-65535
udp   largest_anon_port     rw   65535        --           65535        1024-65535
udp   recv_maxbuf           rw   57344        --           57344        128-1073741824
udp   send_maxbuf           rw   57344        --           57344        1024-1073741824
udp   smallest_anon_port    rw   32768        --           32768        1024-65535
udp   smallest_nonpriv_port rw   1024         --           1024         1024-32768
sctp  extra_priv_ports      rw   2049,4045    --           2049,4045    1-65535
sctp  largest_anon_port     rw   65535        --           65535        1024-65535
sctp  recv_maxbuf           rw   102400       --           102400       8192-1073741824
sctp  send_maxbuf           rw   102400       --           102400       8192-1073741824
sctp  smallest_anon_port    rw   32768        --           32768        1024-65535
sctp  smallest_nonpriv_port rw   1024         --           1024         1024-32768
The first column lists the protocols. Of note is that there are separate IPv4 and IPv6 listings. Per the specification, there is no ttl for IPv6, as is seen by only an IPv4 property. IPv6 calls it the hoplimit, which is more indicative of how the value is actually used.

Including a protocol as an argument lists only those properties.

root@Solaris11Express# ipadm show-prop tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   ecn                   rw   passive      --           passive      never,passive,
                                                                        active
tcp   extra_priv_ports      rw   2049,4045    --           2049,4045    1-65535
tcp   largest_anon_port     rw   65535        --           65535        1024-65535
tcp   recv_maxbuf           rw   128000       --           128000       2048-1073741824
tcp   sack                  rw   active       --           active       never,passive,
                                                                        active
tcp   send_maxbuf           rw   49152        --           49152        4096-1073741824
tcp   smallest_anon_port    rw   32768        --           32768        1024-65535
tcp   smallest_nonpriv_port rw   1024         --           1024         1024-32768
We see the current value, whether we can set it, its default value, and the possible values or range of values. Self documenting. I like it!

To get a specific property, the -p option specifies which one to list.

root@Solaris11Express# ipadm show-prop -p send_maxbuf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   send_maxbuf           rw   49152        --           49152        4096-1073741824
Now to set a property to a specific value, use the format property=value.
root@Solaris11Express# ipadm set-prop -p send_maxbuf=4096 tcp

root@Solaris11Express# ipadm show-prop -p send_maxbuf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   send_maxbuf           rw   4096         4096         49152        4096-1073741824
The value of 4096 in the PERSISTENT column indicates this setting will be retained even after a reboot. To set the property only until the next reboot, use the -t option to set it temporarily.
root@Solaris11Express# ipadm set-prop -t -p send_maxbuf=4096 tcp

root@Solaris11Express# ipadm show-prop -p send_maxbuf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   send_maxbuf           rw   4096         --           49152        4096-1073741824
While it certainly possible to set the value of property back to the same one that is the default, I like the option to set it to its default. This is done with a reset. The PERSISTENT column has reverted back to its original --.
root@Solaris11Express# ipadm reset-prop -p send_maxbuf tcp

root@Solaris11Express# ipadm show-prop -p send_maxbuf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   send_maxbuf           rw   49152        --           49152        4096-1073741824

What About All Those Other ndd Configuration Parameters?

The output of the show-prop operation above is very small compared to what those who use ndd are used to for even just one of the protocols. So what about all the other ndd parameters?

There are two options:

  • Continue to use ndd
  • Use a special parameter conversion of the ndd parameter with ipadm
  • The first is business as usual. The second involves converting the protocol's ndd parameter into one that works with ipadm. The steps that have worked for me are as follows.

    • For any parameter, replace the /dev/protocol and use the protocol as the protocol argument to ipadm. So /dev/tcp becomes tcp.
    • Drop the leading protocol name from the beginning of the parameter, if there is one. So tcp_local_dack_interval becomes _local_dack_interval.
    • If there is no leading procotol name, prepend the property with an underscore (_). For example, tcp_local_dack_interval becomes _tcp_local_dack_interval.
    • For the IP protocol, if there are IPv4 and IPv6 ndd values, indicate the ipadm protocol as ipv4 and ipv6, respectively. With ndd, the lack of a 6 means IPv4.
    Examples of each are as follows.

    Dropping the leading protocol name and specifying it for the protocol argument.

    root@Solaris11Express# ndd -get /dev/tcp tcp_local_dack_interval
    50
    
    root@Solaris11Express# ipadm show-prop -p _local_dack_interval tcp
    PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
    tcp   _local_dack_interval  rw   50           --           50           10-500
    
    Getting a parameter that does not start with the protocol.
    root@Solaris11Express# ndd -get /dev/ip arp_probe_interval
    1500
    
    root@Solaris11Express# ipadm show-prop -p _arp_probe_interval ip
    PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
    ip    _arp_probe_interval   rw   1500         --           1500         10-20000
    
    Distinguishing between IPv4 and IPv6 parameters.
    root@Solaris11Express# ndd -get /dev/ip ip_strict_dst_multihoming
    0
    root@Solaris11Express# ndd -get /dev/ip ip6_strict_dst_multihoming
    0
    
    root@Solaris11Express# ipadm show-prop -p _strict_dst_multihoming ipv4
    PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
    ipv4  _strict_dst_multihoming rw 0            --           0            0-1
    root@Solaris11Express# ipadm show-prop -p _strict_dst_multihoming ipv6
    PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
    ipv6  _strict_dst_multihoming rw 0            --           0            0-1
    
    And when there is an error. All the fields have ? in them.
    root@Solaris11Express# ipadm show-prop -p _strict_dst_multihoming ip
    PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
    ipadm: warning: cannot get property '_strict_dst_multihoming' for 'ip'Unknown property
    ip    _strict_dst_multihoming ?  ?            ?            ?            ?
    
    As more properties are added to ipdam to manage there directly, it will become less necessary to do the ndd work-around.
    Comments:

    Very useful article. Thanks!

    Posted by guest on January 28, 2013 at 10:35 PM EST #

    Post a Comment:
    Comments are closed for this entry.
    About

    stw

    Search

    Archives
    « April 2014
    SunMonTueWedThuFriSat
      
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
       
           
    Today