• March 12, 2015

Routing Leak briefly takes down Google

This morning, users of Google around the world were unable to access many of the company's services due to a routing leak in India. Beginning at 08:58 UTC Indian broadband provider Hathway (AS17488) incorrectly announced over 300 Google prefixes to its Indian transit provider Bharti Airtel (AS9498).

Bharti in turn announced these routes to the rest of the world, and a number of ISPs accepted these routes including US carriers Cogent (AS174), Level 3 (AS3549) as well as overseas incumbent carriers Orange (France Telecom, AS5511), Singapore Telecom (Singtel, AS7473) and Pakistan Telecom (PTCL, AS17557). Like many providers around the world, Hathway peers with Google so that their customers have more direct connectivity with Google services. But when that private relationship enters the public Internet the result can be accidental global traffic redirection.

Last fall, I wrote two blog posts here and here about the issues surrounding routing leaks such this one. Routing leaks happen regularly and can have the effect of misdirecting global traffic. Last month, I gave a talk in the NANOG 63 Peering Forum entitled "Hidden Risks of Peering" that went over some examples of routing leaks like this one.

Below is a graph showing the timeline of the incident for one of the 336 prefixes involved. Bharti (AS9498) should never have been seen as an upstream of Hathway (AS17488) for any Google prefixes. As the graph shows, only a portion of the Internet accepted these routes: the providers who peer with or sell to Bharti, and who failed to filter Bharti's BGP announcements. is a traceroute from one of our servers in Bratislava, Slovakia earlier today showing traffic to Google redirected to India.

trace from Bratislava, Slovakia to (Google) at 09:09 Mar 12, 2015
1  *
2  *
3  *
4      te0-0-2-3.nr11.b027220-0.bts01.atlas.cogentco.com   1.95
5     te0-0-2-0.agr11.bts01.atlas.cogentco.com            1.908
6    te0-3-0-5.ccr21.bts01.atlas.cogentco.com            1.574
7     be2222.ccr21.vie01.atlas.cogentco.com               3.552
8     be2200.ccr21.muc01.atlas.cogentco.com               9.818
9    be2023.ccr21.zrh01.atlas.cogentco.com               14.892
10   be2024.ccr21.mrs01.atlas.cogentco.com               27.371
11                                                        33.255
12   (Airtel Limited, India)                             158.796
13 *
14    (Hathway, Mumbai, India)                            283.586
15 *
16     (Google, Mumbai, India)                             282.664
17   (Google, Mumbai, India)                             294.956
18 *

Highly peered content networks such as Google are uniquely vulnerable to this type of accidental traffic misdirection. Once routes are handed off to a peer, that peer can make a mistake and re-route your traffic. Vigilance is critically important: we know that Hathway was a risky peer for Google because just 22 hours previously, Dyn observed Hathway leaking 134 Google prefixes to Bharti for less than a minute. Careful monitoring of global routing is the only way for enterprises to detect these situations before they become front page news.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha