News, tips, partners, and perspectives for the Oracle Solaris operating system
- November 4, 2020
Snooping IP Interfaces In Oracle Solaris 11.4
Introduction
Since Solaris 11 offers a range of abstractions in the data link layer and IP layer for high availability, scaling and aggregation purposes, It becomes a bit complicated while using snoop command to capture IP or higher-level protocol traffic in case if different layers of physical and logical devices configured in a Solaris system.
Different Abstraction layers provided by Solaris 11
The abstraction provided by different layers are (but not limited to):
- Physical - net0, net1, ixgbe0, ixgbe1 etc
- Data Link - dlmp devices, aggregation (eg: aggr0, aggr1), vnets and vnics
- IP layer - IP interfaces (eg: net0/v4) and IPMP groups(eg: ipmp0/v4addr1)
Using Snoop for high level protocols, snoop provides us option to use a device or IP interface.
From snoop manpage:
snoop [-aqrCDINPSvV] [-t r | a | d] [-c maxcount]
[-d device] [-I IP_interface] [-i filename] [-n filename]
[-o filename] [-p first [, last]] [-s snaplen]
[-x offset [, length]] [expression]
To capture traffic, we can see that we can use one of the following switches:
- snoop -d <device>
- snoop -I <IP Interface>
But let us assume if we are snooping a higher protocols like NFS, SMB or SSH for instance, then, snooping a logical or physical device would not yield a complete capture.
So it is always recommended to use:
- snoop -I <ip interface>
Example Snoop Capture:
Below is a snoop capture of a host that has IPMP configured over two physical interfaces link0_ipmp0 and link1_ipmp0. The IPMP interface name is ipmp0 and the snoop is of an NFSv3 mount operation.
Let us analyze the snoop capture of individual physical interfaces at each layer:
First Physical interface:
Using device link0_ipmp0 (promiscuous mode)
hostA -> hostB PORTMAP C GETPORT prog=100005 (MOUNT) vers=3 proto=UDP
hostB -> hostA PORTMAP R GETPORT port=20048
hostA -> hostB MOUNT3 C Null
hostB -> hostA MOUNT3 R Null
hostB -> hostA RPC R XID=1604329441 Success
...
hostA -> hostB PORTMAP C GETPORT prog=100003 (NFS) vers=3 proto=TCP
...
hostB -> hostA PORTMAP R GETPORT port=2049
...
hostB -> hostA RPC R XID=1604329574 Success
...
hostA -> hostB NFS C FSINFO3 FH=6A8F
hostB -> hostA NFS R FSINFO3 OK
hostA -> hostB NFS C FSSTAT3 FH=6A8F
hostB -> hostA NFS R FSSTAT3 OK
Second Physical interface:
# snoop -d link1_ipmp0 hostB
Using device link1_ipmp0 (promiscuous mode)
hostA -> hostB MOUNT3 C Mount /myshare
...
hostA -> hostB NFS C NULL3
As you can see, the traffic is split between the two interfaces and does not yield the complete picture and is tedious to analyze.
But instead, if we use the ipmp device to snoop, we get a complete capture of the mount operation:
# snoop -I ipmp0 hostB
Using device ipnet/ipmp0 (promiscuous mode)
hostA -> hostB PORTMAP C GETPORT prog=100005 (MOUNT) vers=3 proto=UDP
hostB -> hostA PORTMAP R GETPORT port=20048
hostA -> hostB MOUNT3 C Null
hostB -> hostA MOUNT3 R Null
hostA -> hostB MOUNT3 C Mount /myshare
hostB -> hostA MOUNT3 R Mount OK FH=6A8F Auth=unix
...
hostA -> hostB PORTMAP C GETPORT prog=100003 (NFS) vers=3 proto=TCP
hostB -> hostA PORTMAP R GETPORT port=2049
...
hostA -> hostB NFS C NULL3
...
hostB -> hostA NFS R NULL3
...
hostA -> hostB NFS C FSINFO3 FH=6A8F
hostB -> hostA NFS R FSINFO3 OK
hostA -> hostB NFS C FSSTAT3 FH=6A8F
hostB -> hostA NFS R FSSTAT3 OK
See Also: