In September 2012, CVE-2012-3552 was reported which could allow an
attacker to corrupt slab memory which could lead to a denial-of-service
or possible privilege escalation depending on the target machine
workload. This bug had originally been fixed in the mainline kernel in
April 2011 and was a fairly large patch for a security fix. The RedHat
backport for this fix introduced a new bug which has been assigned
CVE-2013-2224 which again could allow for a denial-of-service or
possible privilege escalation. Rack911 & Tortoiselabs created a
reproducer in June 2013 which would allow an unprivileged user to cause a
RedHat have not yet released a kernel with this CVE fixed, but CentOS have released a custom kernel with the vendor fix for CentOS 6.
We have just released a Ksplice update to address this issue for
releases 5 and 6 of Oracle Linux, RedHat Enterprise Linux, CentOS and
Scientific Linux. We recommend that all users of Ksplice on these
distributions install this zero-downtime update.
If you run a computer network, be it home WiFi or a global enterprise system, you need a way to investigate the machines connected to your network. When ping and traceroute won't cut it, you need a port scanner.
The law and ethics of port scanning are complex. A network scan can be detected by humans or automated systems, and treated as a malicious act, resulting in real costs to the target. Depending on the options you choose, the traffic generated by nmap can range from "completely innocuous" to "watch out for admins with baseball bats". A safe rule is to avoid scanning any network without the explicit permission of its administrators — better yet if that's you.
You'll need root privileges on the scanning system to run most interesting nmap commands, because nmap likes to bypass the standard network stack when synthesizing esoteric packets.
A firm handshake
Let's start by scanning my home network for web and SSH servers:
root@lyle#nmap -sS -p22,80 192.168.1.0/24
Nmap scan report for 192.168.1.1
PORT STATE SERVICE
22/tcp filtered ssh
80/tcp open http
Nmap scan report for 192.168.1.102
PORT STATE SERVICE
22/tcp filtered ssh
80/tcp filtered http
Nmap scan report for 192.168.1.103
PORT STATE SERVICE
22/tcp open ssh
80/tcp closed http
Nmap done: 256 IP addresses (3 hosts up) scanned in 6.05 seconds
We use -p22,80 to ask for a scan of TCP ports 22 and 80, the most popular ports for SSH and web servers respectively. If you don't specify a -p option, nmap will scan the 1,000 most commonly-used ports. You can give a port range like -p1-5000, or even use -p- to scan all ports, but your scan will take longer.
We describe the subnet to scan using CIDR notation. We could equivalently write 192.168.1.1-254.
The option -sS requests a TCP SYN scan. nmap will start a TCP handshake by sending a SYN packet. Then it waits for a response. If the target replies with SYN/ACK, then some program is accepting our connection. A well-behaved client should respond with ACK, but nmap will simply record an open port and move on. This makes an nmapSYN scan both faster and more stealthy than a normal call to connect().
If the target replies with RST, then there's no service on that port, and nmap will record it as closed. Or we might not get a response at all. Perhaps a firewall is blocking our traffic, or the target host simply doesn't exist. In that case the port state is recorded as filtered after nmap times out.
You can scan UDP ports by passing -sU. There's one important difference from TCP: Since UDP is connectionless, there's no particular response required from an open port. Therefore nmap may show UDP ports in the ambiguous state open|filtered, unless you can prod the target application into sending you data (see below).
To save time, nmap tries to confirm that a target exists before performing a full scan. By default it will send ICMP echo (the ubiquitous "ping") as well as TCP SYN and ACK packets. You can use the -P family of options to customize this host-discovery phase.
nmap has the ability to generate all sorts of invalid, useless, or just plain weird network traffic. You can send a TCP packet with no flags at all (null scan, -sN) or one that's lit up "like a Christmas tree" (Xmas scan, -sX). You can chop your packets into little fragments (--mtu) or send an invalid checksum (--badsum). As a network administrator, you should know if the bad guys can confuse your security systems by sending weird packets. As the manpage advises, "Let your creative juices flow".
There's a second benefit to sending weird traffic: We can identify the target's operating system by seeing how it responds to unusual situations. nmap will perform this OS detection if you specify the -O flag:
root@lyle#nmap -sS -O 192.168.1.0/24
Nmap scan report for 192.168.1.1
Not shown: 998 filtered ports
PORT STATE SERVICE
23/tcp closed telnet
80/tcp open http
MAC Address: 00:1C:10:33:6B:99 (Cisco-Linksys)
Device type: WAP|broadband router
Running: Linksys embedded, Netgear embedded, Netgear VxWorks 5.X
Nmap scan report for 192.168.1.100
Not shown: 998 filtered ports
PORT STATE SERVICE
139/tcp open netbios-ssn
445/tcp open microsoft-ds
MAC Address: 00:1F:3A:7F:7C:26 (Hon Hai Precision Ind.Co.)
Warning: OSScan results may be unreliable because we could not find
at least 1 open and 1 closed port
Device type: general purpose
Running (JUST GUESSING) : Microsoft Windows Vista|2008|7 (98%)
Nmap scan report for 192.168.1.104
All 1000 scanned ports on 192.168.1.104 are closed
MAC Address: 7C:61:93:53:9F:E5 (Unknown)
Too many fingerprints match this host to give specific OS details
Since the first target has both an open and a closed port, nmap has many protocol corner cases to explore, and it easily recognizes a Linksys home router. With the second target, there's no port in the closed state, so nmap isn't as confident. It guesses a Windows OS, which seems especially plausible given the open NetBIOS ports. In the last case nmap has no clue, and gives us some raw findings only. If you know the OS of the target, you can contribute this fingerprint and help make nmap even better.
Behind the port
It's all well and good to discover that port 1234 is open, but what's actually listening there? nmap has a version detection subsystem that will spam a host's open ports with data in hopes of eliciting a response. Let's pass -sV to try this out:
root@lyle#nmap -sS -sV 192.168.1.117
Nmap scan report for 192.168.1.117
Not shown: 998 closed ports
PORT STATE SERVICE VERSION
443/tcp open ssh OpenSSH 5.5p1 Debian 6 (protocol 2.0)
8888/tcp open http thttpd 2.25b 29dec2003
nmap correctly spotted an HTTP server on non-standard port 8888. The SSH server on port 443 (usually HTTPS) is also interesting. I find this setup useful when connecting from behind a restrictive outbound firewall. But I've also had network admins send me worried emails, thinking my machine has been compromised.
nmap also gives us the exact server software versions, straight from the server's own responses. This is a great way to quickly audit your network for any out-of-date, insecure servers.
Since a version scan involves sending application-level probes, it's more intrusive and can cause more trouble. From the book:
In the nmap-service-probes included with Nmap the only ports excluded are TCP port 9100 through 9107. These are common ports for printers to listen on and they often print any data sent to them. So a version detection scan can cause them to print many pages full of probes that Nmap sends, such as SunRPC requests, help statements, and X11 probes.
This behavior is often undesirable, especially when a scan is meant to be stealthy.
Trusting the source
It's a common (if questionable) practice for servers or firewalls to trust certain traffic based on where it appears to come from. nmap gives you a variety of tools for mapping these trust relationships. For example, some firewalls have special rules for traffic originating on ports 53, 67, or 20. You can set the source port for nmap's TCP and UDP packets by passing --source-port.
You can also spoof your source IP address using -S, and the target's responses will go to that fake address. This normally means that nmap won't see any results. But these responses can affect the unwitting source machine's IP protocol state in a way that nmap can observe indirectly. You can read about nmap's TCP idle scan for more details on this extremely clever technique. Imagine making any machine on the Internet — or your private network — port-scan any other machine, while you collect the results in secret. Can you use this to map out trust relationships in your network? Could an attacker?
You pull out your laptop and type http://www.google.com into the URL bar on your browser. Instead of your friendly search box, you get a page where you pay money or maybe watch an advertisement, agree to some terms of service, and are only then free to browse the web.
What is going on behind the scenes to give the coffee shop that kind of control over your packets? Let's trace an example of that process from first broadcast to last redirect and find out.
Step 1: Get our network configuration
When I first sit down and turn on my laptop, it needs to get some network information and join a wireless network.
This laptop happens to use the DCHP client dhclient. /etc/dhcp3/dhclient.conf is a sample dhclient configuration file describing among other things what the client will request from a DHCP server (your network manager might frob that configuration -- on my Ubuntu laptop, NetworkManager keeps a modified config at /var/run/nm-dhclient-wlan0.conf).
A DHCP negotiation happens in 4 parts:
Step 1: DHCP discovery. The DHCP client (us, 0.0.0.0 in the screencap) sends a message to Ethernet broadcast address ff:ff:ff:ff:ff:ff to discover DHCP servers (Wireshark shows IP addresses in the summary view, so we see broadcast IP address 255.255.255.255). The packet includes a parameter request list with the parameters in the dhclient config file. The parameters in my /var/run/nm-dhclient-wlan0.conf are:
Step 2: DHCP offer. DHCP servers that get the discovery broadcast allocate an IP address and respond with a DHCP broadcast containing that IP address and other lease information. This is typically a simple race -- whoever gets an offer packet to the requester first wins. In our case, only MAC address00:90:fb:17:ca:4e (Wireshark shows IP address 192.168.5.1) answers our discovery broadcast.
Step 3: DHCP request. The DHCP client picks an offer and sends another DHCP broadcast, informing the DHCP servers of the winner and letting the losers de-allocate their reserved IP addresses.
Step 4: DHCP acknowledgment. The winning DHCP server acknowledges completion of the DHCP exchange and reiterates the DHCP lease parameters. We now have an IP address (192.168.5.87) and know the IP address of our gateway router (192.168.5.1):
Step 2: Find our gateway
We managed to get a lot done using broadcast packets, but at this point a) nobody in our broadcast domain knows our MAC address, and b) we don't know the MAC address of our gateway, so we can't get any packets routed out to the Internet. Let's fix that:
Before offering us IP address 192.168.5.87, the DHCP server (Portwell_17:ca:4) sends an ARP request for that address, saying "Who has 192.168.5.87. If that's you, respond with your MAC address". Since nobody answers, the server can be fairly confident that the IP address is not already in use.
After getting assigned IP address 192.168.5.87, we (Apple_8f:95:3f) double-check that nobody else is using it with a few ARP requests that nobody answers. We then send a few gratuitous ARPs to let everyone know that it's ours and they should update their ARP caches.
We then make an ARP request for the MAC address corresponding to the IP address for our gateway router: 192.168.5.1. Our DHCP server happens to also be our gateway router and responds claiming the address.
Step 3: Get past the terms of service
Now that we have an IP address and know the IP address of our gateway, we should be able to send packets through our gateway and out to the Internet.
Looking back at the DHCP acknowledgement, as part of the lease we were given a domain name: sbx07338.cambrma.wayport.net. What our local DNS resolver decides to do with host www.google.com, since it potentially isn't fully-qualified, is append the domain name from the DHCP lease to it (eg www.google.com.sbx07338.cambrma.wayport.net in the first iteration) and try to resolve that. When the resolution fails, it tries appending decreasingly specific parts of the DHCP domain name, finds that all of them fail, and then gives up and tries to resolve plain old www.google.com. This works, and we get back IP address 188.8.131.52. A whois after the fact confirms that this is Google:
jesstess@pretzel-logic:~$ whois 184.108.40.206
NetRange: 220.127.116.11 - 18.104.22.168
OrgName: Google Inc.
Address: 1600 Amphitheatre Parkway
City: Mountain View
We complete a TCP handshake with ``22.214.171.124'' and make an HTTP GET request for the resource we want (/). Instead of getting back an HTTP 200 and the Google home page, we receive a 302 redirect to http://nmd.sbx07338.cambrma.wayport.net/index.adp?
Our MAC address and IP address are conveniently encoded in the redirect URL.
So what is going on here? Why didn't we get back the Google home page?
Our DHCP server/router, 192.168.5.1, is capturing our HTTP traffic and redirecting it to a special landing page. We don't get to make it out to Google until we finish playing a game with the coffee shop.
Let's dwell on this for a moment, because it's kind of amazing that the way the Internet is designed, our gateway router can hijack our HTTP requests and we can't stop it. In this case, we can see that the URL has changed in our browser after the redirect, but if a malicious gateway were transparently proxying our HTTP requests to an evil malware-laden clone of www.google.com, we'd have no way to notice because there wouldn't be a redirect and the URL wouldn't change.
Worrisome? Definitely, if you're trusting a gateway with sensitive information. If you don't want to have to trust your gateway, you have to use point-to-point encryption: HTTPS, SSH, your favorite IPSec or SSLVPN, etc. And then hope there aren't bugs in your secure protocol's implementation.
Well, ain't nothing to it but to do a DNS lookup on the host name in the redirect (nmd.sbx07338.cambrma.wayport.ne) and make our request there:
nmd is a host in the domain from our DHCP lease, so our local resolver's rules manage to resolve it in one try, and we get IP address 126.96.36.199. This is incidentally the IP address of the DHCP Server Identifier we received with our DHCP lease.
We try our HTTP GET request again there and get back an HTTP 200 and a landing page (still not the Google home page), which the browser renders.
The landing page has some ads and terms of service, and a button to click that we're told will grant us Internet access. That click generates an HTTP POST:
Step 4: Get to Google
Having agreed to the terms of service, 188.8.131.52 communicates to our gateway router that our MAC address (which was passed in the redirect URL) should be added to a whitelist, and our traffic shouldn't be captured and redirected anymore -- our HTTP packets should be allowed out onto the Internet.
184.108.40.206 responds to our POST with a final HTTP 302 redirect, this time to starbucks.yahoo.com. We do a final DNS lookup, make our HTTP GET, and get served an HTTP 200 and a webpage with some ads enticing us to level up our coffee addiction. We now have real Internet access and can get to http://www.google.com.
And that's the story! This ability to hijack HTTP traffic at a gateway until terms are met has over the years facilitated a huge industry based around private WiFi networks at coffee shops, airports, and hotels.
It is also a reminder about just how much control your gateway, or a device pretending to be your gateway, has when you use insecure protocols. Upside-down-ternet is a playful example of exploiting the trust in your gateway, but bogus DNS resolution or transparently proxying requests to malicious sites makes use of this same principle.
ARP comes into play when you, for example, head over to a friend's house, pull out your laptop, and try to use the wireless to surf the web. One of the first things that probably needs to happen is determining the MAC address of the gateway (probably your friend's router), so that the Ethernet packets containing all those IP[TCP[HTTP]] requests you want to send out to the Internet know how to get to their first hop, the gateway.
Your laptop finds out the MAC address of the gateway by asking. It broadcasts an ARP request for "Who has IP address 192.168.1.1", and the gateway broadcasts an ARP response saying "I have 192.168.1.1, and my MAC address is xx:xx:xx:xx:xx:xx". Your laptop, armed with the MAC address of the gateway, can then craft Ethernet packets that will go to the gateway and get routed out to the Internet.
But the gateway didn't really have to prove who it was. It just asserted who it was, and everyone listened. Anyone else can send an ARP response claiming to have IP address 192.168.1.1. And that's the ticket: if you can pretend to be the gateway, you can control all the packets that get routed through the gateway and the content returned to clients.
Step 1: The layout
I did this at home. The three machines involved were:
real gateway router: IP address 192.168.1.1, MAC address 68:7f:74:9a:f4:ca
fake gateway: a desktop called kid-charlemagne, IP address 192.168.1.200, MAC address 00:30:1b:47:f2:74
test machine getting duped: a laptop on wireless called pixeleen, IP address 192.168.1.111, MAC address 00:23:6c:8f:3f:95
The gateway router, like most modern routers, is bridging between the wireless and wired domains, so ARP packets get broadcast to both domains.
Step 2: Enable IPv4 forwarding
kid-charlemagne wants to be receiving packets that aren't destined for it (eg the web traffic). Unless IP forwarding is enabled, the networking subsystem is going to ignore packets that aren't destined for us. So step 1 is to enable IP forwarding. All that takes is a non-zero value in /proc/sys/net/ipv4/ip_forward:
Step 3: Set routing rules so packets going through the gateway get routed to you
kid-charlemagne is going to act like a little NAT. For HTTP packets heading out to the Internet, kid-charlemagne is going to rewrite the destination address in the IP packet headers to be its own IP address, so it becomes final destination for the web traffic:
For HTTP packets heading back from kid-charlemagne to the client, it'll rewrite the source address to be that of the original destination out on the Internet.
We can set up this routing rule with the following iptables command:
The actual rule (-t nat ... -j NETMAP --to 192.168.1.200)
-t says we're specifying a table. The nat table is where a lookup happens on packets that create new connections. The nat table comes with 3 built-in chains: PREROUTING, OUTPUT, and POSTROUTING. We want to add a rule in the PREROUTING chain, which will alter packets right as they come in, before routing rules have been applied.
That PREROUTING rule is going to apply to TCP packets destined for port 80 (-p tcp --dport 80), aka HTTP traffic. For packets that match this filter, jump (-j) to the following action:
If we receive a packet heading for some destination, rewrite the destination in the IP header to be 192.168.1.200 (NETMAP --to 192.168.1.200). Have the nat table keep a mapping between the original destination and rewritten destination. When a packet is returning through us to its source, rewrite the source in the IP header to be the original destination.
In summary: "If you're a TCP packet destined for port 80 (HTTP traffic), actually make my address, 192.168.1.200, the destination, NATting both ways so this is transparent to the source."
One last thing:
The networking subsystem will not allow you to ARP for a random IP address on an interface -- it has to be an IP address actually assigned to that interface, or you'll get a bind error along the lines of "Cannot assign requested address". We can handle this by adding an ip entry on the interface that is going to send packets to pixeleen, the test client. kid-charlemagne is wired, so it'll be eth0.
jesstess@kid-charlemagne:~$ sudo ip addr add 192.168.1.1/24 dev eth0
We can check our work by listing all our interfaces' addresses and noting that we now have two IP addresses for eth0, the original IP address 192.168.1.200, and the gateway address 192.168.1.1.
jesstess@kid-charlemagne:~$ ip addr
3: eth0: mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:30:1b:47:f2:74 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.200/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.1/24 scope global secondary eth0
inet6 fe80::230:1bff:fe47:f274/64 scope link
valid_lft forever preferred_lft forever
Step 4: Set yourself up to respond to HTTP requests
kid-charlemagne happens to have Apache set up. You could run any minimalist web server that would, given a request for an arbitrary resource, do something interesting.
Step 5: Test pretending to be the gateway
At this point, kid-charlemagne is ready to pretend to be the gateway. The trouble is convincing pixeleen that the MAC address for the gateway has changed, to that of kid-charlemagne. We can do this by sending a Gratuitous ARP, which is basically a packet that says "I know nobody asked, but I have the MAC address for 192.168.1.1”. Machines that hear that Gratuitous ARP will replace an existing mapping from 192.168.1.1 to a MAC address in their ARP caches with the mapping advertised in that Gratuitous ARP.
We can look at the ARP cache on pixeleen before and after sending the Gratuitous ARP to verify that the Gratuitious ARP is working.
pixeleen’s ARP cache before the Gratuitous ARP:
jesstess@pixleen$ arp -a
? (192.168.1.1) at 68:7f:74:9a:f4:ca on en1 ifscope [ethernet]
? (192.168.1.200) at 0:30:1b:47:f2:74 on en1 ifscope [ethernet]
68:7f:74:9a:f4:ca is the MAC address of the real gateway router.
There are lots of command line utilities and bindings in various programming language that make it easy to issue ARP packets. I used the arping tool:
jesstess@kid-charlemagne:~$ sudo arping -c 3 -A -I eth0 192.168.1.1
We'll send a Gratuitous ARP reply (-A), three times (-c -3), on the eth0 interface (-l eth0) for IP address 192.168.1.1.
As soon as we generate the Gratuitous ARPs, if we check pixeleen’s ARP cache:
jesstess@pixeleen$ arp -a
? (192.168.1.1) at 0:30:1b:47:f2:74 on en1 ifscope [ethernet]
? (192.168.1.200) at 0:30:1b:47:f2:74 on en1 ifscope [ethernet]
Bam. pixeleen now thinks the MAC address for IP address 220.127.116.11 is 0:30:1b:47:f2:74, which is kid-charlemagne’s address.
If I try to browse the web on pixeleen, I am served the resource matching the rules in kid-charlemagne’s web server.
We can watch this whole exchange in Wireshark:
First, the Gratuitous ARPs generated by kid-charlemagne:
The only traffic getting its headers rewritten so that kid-charlemagne is the destination is HTTP traffic: TCP traffic on port 80. That means all of the non-HTTP traffic associated with viewing a web page still happens as normal. In particular, when kid-charlemagne gets the DNS resolution requests for lycos.com, the test site I visited, it will follow its routing rules and forward them to the real router, which will send them out to the Internet:
The HTTP traffic gets served by kid-charlemagne:
Note that the HTTP request has a source IP of 192.168.1.111, pixeleen, and a destination IP of 18.104.22.168, which dig -x 22.214.171.124 +short tells us is search-core1.bo3.lycos.com. The HTTP response has a source IP of 126.96.36.199 and a destination IP of 192.168.1.111. The fact that kid-charlemagne has rerouted and served the request is totally transparent to the client at the IP layer.
Step 6: Deploy against friends and family
I trust you to get creative with this.
Step 7: Reset everything to the normal state
To get the normal gateway back in control, delete the IP address from the interface on kid-charlemagne and delete the iptables routing rule:
To get the client machines to believe the router is the real gateway, you might have to clear the gateway entry from the ARP cache with arp -d 192.168.1.1, or bring your interfaces down and back up. I can verify that my TiVo corrected itself quickly without any intervention, but I won't make any promises about your networked devices.
That was a lot of explanatory text, but the steps required to hijack the HTTP traffic on your home subnet can be boiled down to:
enabled IP forwarding: echo 1 > /proc/sys/net/ipv4/ip_forward
set your routing rule: iptables -t nat -A PREROUTING -p tcp --dport 80 -j NETMAP --to 192.168.1.200
add the gateway IP address to the appropriate interface: ip addr add 192.168.1.1/24 dev eth0
ARP for the gateway MAC address: arping -c 3 -A -I eth0 192.168.1.1
substituting the appropriate IP address and interface information and tearing down when you're done.
And that's all there is to it!
This has been tested as working in a few environments, but it might not work in yours. I'd love to hear the details on if this works, works with modifications, or doesn't work (because the devices are being too clever about Gratuitous ARPs, or otherwise) in the comments.
--> Huge thank-you to fellow experimenter adamf. <---
Anyone who administers even a moderately sized network knows that when problems arise, diagnosing and fixing them can be extremely difficult. They're usually non-deterministic and difficult to reproduce, and very similar symptoms (e.g. a slow or unreliable connection) can be caused by any number of problems — congestion, a broken router, a bad physical link, etc.
One very useful weapon in a system administrator's arsenal for dealing with network issues is traceroute (or tracert, if you use Windows). This is a neat little program that will print out the path that packets take to get from the local machine to a destination — that is, the sequence of routers that the packets go through.
Using traceroute is pretty straightforward. On a UNIX-like system, you can do something like the following:
$ traceroute google.com
traceroute to google.com (188.8.131.52), 30 hops max, 60 byte packets
1 router.lan (192.168.1.1) 0.595 ms 1.276 ms 1.519 ms
2 184.108.40.206 (220.127.116.11) 13.669 ms 17.583 ms 18.242 ms
3 ge-2-20-ur01.cambridge.ma.boston.comcast.net (18.104.22.168) 18.710 ms 19.192 ms 19.640 ms
4 be-51-ar01.needham.ma.boston.comcast.net (22.214.171.124) 20.642 ms 21.160 ms 21.571 ms
5 pos-2-4-0-0-cr01.newyork.ny.ibone.comcast.net (126.96.36.199) 28.870 ms 29.788 ms 30.437 ms
6 pos-0-3-0-0-pe01.111eighthave.ny.ibone.comcast.net (188.8.131.52) 30.911 ms 17.377 ms 15.442 ms
7 as15169-3.111eighthave.ny.ibone.comcast.net (184.108.40.206) 40.081 ms 41.018 ms 39.229 ms
8 220.127.116.11 (18.104.22.168) 20.139 ms 21.629 ms 20.965 ms
9 22.214.171.124 (126.96.36.199) 25.771 ms 26.196 ms 26.633 ms
10 188.8.131.52 (184.108.40.206) 23.856 ms 24.820 ms 27.722 ms
Pretty nifty. But how does it work? After all, when a packet leaves your network, you can't monitor it anymore. So when it hits all those routers, the only way you can know about that is if one of them tells you about it.
The secret behind traceroute is a field called "Time To Live" (TTL) that is contained in the headers of the packets sent via the Internet Protocol. When a host receives a packet, it checks if the packet's TTL is greater than 1 before sending it on down the chain. If it is, it decrements the field. Otherwise, it drops the packet and sends an ICMPTIME_EXCEEDED packet to the sender. This packet, like all IP packets, contains the address of its sender, i.e. the intermediate host.
traceroute works by sending consecutive requests to the same destination with increasing TTL fields. Most of these attempts result in messages from intermediate hosts saying that the packet was dropped. The IP addresses of these intermediate hosts are then printed on the screen (generally with an attempt made at determining the hostname) as they arrive, terminating when the maximum number of hosts have been hit (on my machine's traceroute the default maximum is 30, but this is configurable), or when the intended destination has been reached.
The rest of this post will walk through implementing a very primitive version of traceroute in Python. The real traceroute is of course more complicated than what we will create, with many configurable features and modes. Still, our version will implement the basic functionality, and at the end, we'll have a really nice and short Python script that will do just fine for performing a simple traceroute.
So let's begin. Our algorithm, at a high level, is an infinite loop whose body creates a connection, prints out information about it, and then breaks out of the loop if a certain condition has been reached. So we can start with the following skeletal code:
defmain(dest):whileTrue:# ... open connections ...# ... print data ...# ... break if useful ...passif__name__=="__main__":main('google.com')
The socket module provides a gethostbyname() method that attempts to resolve a domain name into an IP address:
#!/usr/bin/pythonimportsocketdefmain(dest_name):dest_addr=socket.gethostbyname(dest_name)whileTrue:# ... open connections ...# ... print data ...# ... break if useful ...passif__name__=="__main__":main('google.com')
Step 2: Create sockets for the connections.
We'll need two sockets for our connections — one for receiving data and one for sending. We have a lot of choices for what kind of probes to send; let's use UDP probes, which require a datagram socket (SOCK_DGRAM). The routers along our traceroute path are going to send back ICMP packets, so for those we need a raw socket (SOCK_RAW).
#!/usr/bin/pythonimportsocketdefmain(dest_name):dest_addr=socket.gethostbyname(dest_name)icmp=socket.getprotobyname('icmp')udp=socket.getprotobyname('udp')whileTrue:recv_socket=socket.socket(socket.AF_INET,socket.SOCK_RAW,icmp)send_socket=socket.socket(socket.AF_INET,socket.SOCK_DGRAM,udp)# ... print data ...# ... break if useful ...if__name__=="__main__":main('google.com')
Step 3: Set the TTL field on the packets.
We'll simply use a counter which begins at 1 and which we increment with each iteration of the loop. We set the TTL using the setsockopt module of the socket object:
#!/usr/bin/pythonimportsocketdefmain(dest_name):dest_addr=socket.gethostbyname(dest_name)icmp=socket.getprotobyname('icmp')udp=socket.getprotobyname('udp')ttl=1whileTrue:recv_socket=socket.socket(socket.AF_INET,socket.SOCK_RAW,icmp)send_socket=socket.socket(socket.AF_INET,socket.SOCK_DGRAM,udp)send_socket.setsockopt(socket.SOL_IP,socket.IP_TTL,ttl)ttl+=1# ... print data ...# ... break if useful ...if__name__=="__main__":main('google.com')
Step 4: Bind the sockets and send some packets.
Now that our sockets are all set up, we can put them to work! We first tell the receiving socket to listen to connections from all hosts on a specific port (most implementations of traceroute use ports from 33434 to 33534 so we will use 33434 as a default). We do this using the bind() method of the receiving socket object, by specifying the port and an empty string for the hostname. We can then use the sendto() method of the sending socket object to send to the destination host (on the same port). The first argument of the sendto() method is the data to send; in our case, we don't actually have anything specific we want to send, so we can just give the empty string:
#!/usr/bin/pythonimportsocketdefmain(dest_name):dest_addr=socket.gethostbyname(dest_name)port=33434icmp=socket.getprotobyname('icmp')udp=socket.getprotobyname('udp')ttl=1whileTrue:recv_socket=socket.socket(socket.AF_INET,socket.SOCK_RAW,icmp)send_socket=socket.socket(socket.AF_INET,socket.SOCK_DGRAM,udp)send_socket.setsockopt(socket.SOL_IP,socket.IP_TTL,ttl)recv_socket.bind(("",port))send_socket.sendto("",(dest_name,port))ttl+=1# ... print data ...# ... break if useful ...if__name__=="__main__":main('google.com')
Step 5: Get the intermediate hosts' IP addresses.
Next, we need to actually get our data from the receiving socket. For this, we can use the recvfrom() method of the object, whose return value is a tuple containing the packet data and the sender's address. In our case, we only care about the latter. Note that the address is itself actually a tuple containing both the IP address and the port, but we only care about the former. recvfrom() takes a single argument, the blocksize to read — let's go with 512.
It's worth noting that some administrators disable receiving ICMP ECHO requests, pretty much specifically to prevent the use of utilities like traceroute, since the detailed layout of a network can be sensitive information (another common reason to disable them is the ping utility, which can be used for denial-of-service attacks). It is therefore completely possible that we'll get a timeout error, which will result in an exception. Thus, we'll wrap this call in a try/except block. Traditionally, traceroute prints asterisks when it can't get the address of a host. We'll do the same once we print out results.
#!/usr/bin/pythonimportsocketdefmain(dest_name):dest_addr=socket.gethostbyname(dest_name)port=33434icmp=socket.getprotobyname('icmp')udp=socket.getprotobyname('udp')ttl=1whileTrue:recv_socket=socket.socket(socket.AF_INET,socket.SOCK_RAW,icmp)send_socket=socket.socket(socket.AF_INET,socket.SOCK_DGRAM,udp)send_socket.setsockopt(socket.SOL_IP,socket.IP_TTL,ttl)recv_socket.bind(("",port))send_socket.sendto("",(dest_name,port))curr_addr=Nonetry:_,curr_addr=recv_socket.recvfrom(512)curr_addr=curr_addrexceptsocket.error:passfinally:send_socket.close()recv_socket.close()ttl+=1# ... print data ...# ... break if useful ...if__name__=="__main__":main('google.com')
Step 6: Turn the IP addresses into hostnames and print the data.
To match traceroute's behavior, we want to try to display the hostname along with the IP address. The socket module provides the gethostbyaddr() method for reverse DNS resolution. The resolution can fail and result in an exception, in which case we'll want to catch it and make the hostname the same as the address. Once we get the hostname, we have all the information we need to print our data:
#!/usr/bin/pythonimportsocketdefmain(dest_name):dest_addr=socket.gethostbyname(dest_name)port=33434icmp=socket.getprotobyname('icmp')udp=socket.getprotobyname('udp')ttl=1whileTrue:recv_socket=socket.socket(socket.AF_INET,socket.SOCK_RAW,icmp)send_socket=socket.socket(socket.AF_INET,socket.SOCK_DGRAM,udp)send_socket.setsockopt(socket.SOL_IP,socket.IP_TTL,ttl)recv_socket.bind(("",port))send_socket.sendto("",(dest_name,port))curr_addr=Nonecurr_name=Nonetry:_,curr_addr=recv_socket.recvfrom(512)curr_addr=curr_addrtry:curr_name=socket.gethostbyaddr(curr_addr)exceptsocket.error:curr_name=curr_addrexceptsocket.error:passfinally:send_socket.close()recv_socket.close()ifcurr_addrisnotNone:curr_host="%s (%s)"%(curr_name,curr_addr)else:curr_host="*"print"%d\t%s"%(ttl,curr_host)ttl+=1# ... break if useful ...if__name__=="__main__":main('google.com')
Step 7: End the loop.
There are two conditions for exiting our loop — either we have reached our destination (that is, curr_addr is equal to dest_addr)1 or we have exceeded some maximum number of hops. We will set our maximum at 30:
Hurrah! The data matches the real traceroute's perfectly.
Of course, there are many improvements that we could make. As I mentioned, the real traceroute has a whole slew of other features, which you can learn about by reading the manpage. In the meantime, I wrote a slightly more complete version of the above code that allows configuring the port and max number of hops, as well as specifying the destination host. You can download it at my github repository.
Alright folks, What UNIX utility should we write next? strace, anyone? :-) 2
1 This is actually not quite how the real traceroute works. Rather than checking the IP addresses of the hosts and stopping when the destination address matches, it stops when it receives a ICMP "port unreachable" message, which means that the host has been reached. For our purposes, though, this simple address heuristic is good enough.
2 Ksplice blogger Nelson took up a DIY strace on his personal blog, Made of Bugs.
Wireless traffic is both interesting and delightfully accessible thanks to the broadcast nature of 802.11. I have spent many a lazy weekend afternoon watching my laptop, the Tivo, and the router chatting away in a Wireshark window.
As fun as the wireless traffic in one's house may be, there's something to be said for being able to observe a much larger ecosystem - one with more people with a more diverse set of operating systems, browsers, and intentions as they work on their wireless-enabled devices, giving rise to more interesting background and active traffic patterns and a greater set of protocols in play.
Now, it happens to be the case that sniffing other people's wireless traffic breaks a number of federal and local laws, including wiretapping laws, and while I am interested in observing other people's wireless traffic, I'm not interested in breaking the law. Fortunately, Ksplice is down the road from a wonderful school that fosters this kind of intellectual curiosity.
Some interesting results from the data set collected are summarized below. Traffic was gathered with tcpdump on my laptop as I sat in the middle of the classroom. The data was imported into Wireshark, spit back out as a CSV, and imported into a sqlite database for aggregation queries, read back into tcpdump and filtered there, or hacked up with judicious use of grep, as different needs arose.
Basic statistics Time spent capturing: 45 minutes Packets captured: 853436 Number of traffic sources in the room: 21 Number of distinct source and destination IPv4 and IPv6 addresses: 5117 Number of "active" traffic addresses (eg using HTTP, SSH, not background traffic): 581 Number of protocols represented: 48 (note that Wireshark buckets based on the top layer for a packet, so for example TCP is in this count because someone was sending actual TCP traffic without an application layer on top and not because TCP is the transport protocol for HTTP, which is also in this count). These protocols and how much traffic was sent over them are in the table on the left.
IPv4 v. IPv6 Number of IPv4 packets: 580547 Number of IPv6 packets: 270351
2.15 IPv4 packets were sent for every IPv6 packet. IPv6 was only used for background traffic, serving as the internet layer for the following protocols: DHCPv6, DNS, ICMPv6, LLMNR, MDNS, SSDP, TCP, and UDP. The TCP over IPv6 packets were all icslap, ldap, or wsdapi communications between our Windows user discussed below and his or her remote desktop. The UDP over IPv6 packets were all ws-discovery communications, part of a local multicast discovery protocol most likely being used by the Windows machines in the room.
ICMP v. ICMPv6 Number of ICMP packets: 60 Number of ICMPv6 packets: 116167
1936.12 ICMPv6 packets were sent for every ICMP packet. The reason is that ICMPv6 is doing IPv6 work that is taken care of by other link layer protocols in IPv4. Looking at the ICMP and ICMPv6 packets by type:
Dest unreachable (Host administratively prohibited)
Dest unreachable (Port unreachable)
Echo (ping) request
Time-to-live exceeded in transit
Multicast Listener Report msg v2
Multicast listener done
Multicast listener report
Time exceeded (In-transit)
Unknown (0x40) (Unknown (0x87))
Unreachable (Port unreachable)
These ICMPv6 packets are mostly doing Neighbor Discover Protocol (NDP) and Multicast Listener Discovery (MLD) work. NDP handles router and neighbor solicitation and is similar to ARP and ICMP under IPv4, and MLD handles multicast listener discovery similar to IGMP under IPv4.
TCP v. UDP Number of TCP packets: 383122 Number of UDP packets: 350067
1.09 TCP packets were sent for every UDP packet. I would have thought TCP would be a clear winner, but given that MDNS traffic, which is over UDP, makes up over 30% of the packets captured, I guess this isn't surprising. The 14% of packets unaccounted for at the transport layer are mostly ARP and ICMP traffic. See also this post.
Awesomely, AIM, Jabber, MSN Messenger, and Yahoo! Messenger were all represented in the traffic:
AIM is the clear favorite (at least with this small sample size). Note that Jabber has about 1/4th the AIM participants but 4x the number of packets. Either the Jabberers are extra chatty, or the fact that Jabber is an XML-based protocol inflates the size of a conversation dramatically on the wire. Note that some IM traffic (like Google Chat) might have instead been bucketed as HTTP/XML by Wireshark.
That Windows Remote Desktop Person
119489 packets, or 14% of the traffic, were between a computer in the classroom and what is with high probability a Windows machine on campus running the Microsoft Remote Desktop Protocol (see also this support page for a discussion of the protocol specifics).
RDP traffic from client to remote desktop: T.125, TCP, TPKT, X.224 RDP traffic from remote desktop to client: TCP, TPKT
Most of the traffic is T.125 payload packets. TPKTs encapulate transport protocol data units (TPDUs) in the ISO Transport Service on top of TCP. TPKT traffic was all "Continuation" traffic. X.224 transmits status and error codes. TCP "ms-wbt-server" traffic to port 3389 on the remote machine seals the deal on this being an RDP setup.
All SSH and SSHv2 traffic was to Linerva, a public-access Linux server run by SIPB for the MIT community, except for one person talking to a personal server on campus.
5 clients were involved with negotiations with SSLv2, which is insecure and disabled by default on most browsers and never got past a "Client Hello".
I wanted to be able to answer questions like "what were the top 20 most visited website" in this traffic capture. The proliferation of content distribution networks makes it harder to track all traffic associated with a popular website by IP addresses or hostnames. I ended up doing a crude but quick grep "Host:" pcap.plaintext | sort | uniq -c | sort -n -r on the expanded plaintext version of the data exported from Wireshark, which gives the most visited hosts based on the number of GET requests. The top 20 most visited hosts by that method were:
Alas, pretty boring. The blogcdn, blogsmith and eyewonder hosts are all for Engadget, and fbcdn is part of Facebook. I'll admit that I'd been a little hopeful that some enterprising student would try to screw up my data by scripting visits to a bunch of porn sites or something. CDNs dominate the top 20, and in fact almost all of the roughly 410 web server IP addresses gathered are for CDNs. Akamai led with 39 distinct IPs, followed by Amazon AWS with 23, and Facebook and Panther CDN with 16, with many more at lower numbers.
Using the Internet means a lot more than HTTP traffic! 45 minutes of traffic capture gave us 48 protocols to explore. Most of the captured traffic was background traffic, and in particular discovery traffic local to the subnet.
Sniffing wireless traffic (legally) is a great way to learn about networking protocols and the way the Internet works in practice. It can also be incredibly useful for debugging networking problems.
What are some of your fun or awful networking experiences?
Tired of rebooting to update systems? So are we -- which is why we invented Ksplice, technology that lets you update the Linux kernel without rebooting. It's currently available as part of Oracle Linux Premier Support, Fedora, and Ubuntu desktop. This blog is our place to ramble about technical topics that we (and hopefully you) think are interesting.