Report GoogleBot, Yahoo!Slurp and MSNBot in Client Named Locations

Dear Reader,

I am sometimes approached by customers wanting to identify traffic based on the client IP origin especially for the Search Bots of the three most common providers in the market (GoogleBot, MSNBot, Yahoo!Slurp). Based on the IP address information I found on the (http://chceme.info/ips/) Website I have setup the following network ranges in a single file to upload to RUEI.
Generic KPI with metric Throughput (KBps) and filters on all three Client Named Robots.

Simply go to your RUEI » Configuration » General » Named clients and select the Upload button from the toolbar to merge the entries of the client_named_robots.txt file (i.e. UTF-8) with your existing Client Named Locations. You can then start to filter out this content from your Real User Experience Insight Analysis Data & KPI's using the Client Named Groups/Networks as detailled above.

Please let me know good uses you can make of this and alterations or additions which are suitable for your work.
Kind regards,
Stefan



# client_named_robots.txt
# http://blogs.oracle.com/sthieme/resource/client_named_robots.txt
#
# IP base address IP netmask Client Named Group Client Named Network
64.233.160.0 255.255.224.0 Google (Googlebot) Googlebot (64.233.160.0/19)
64.102.0.0 255.255.240.0 Google (Googlebot) Googlebot (66.102.0.0/20)
64.249.64.0 255.255.224.0 Google (Googlebot) Googlebot (66.249.64.0/19)
72.14.192.0 255.255.192.0 Google (Googlebot) Googlebot (72.14.192.0/18)
72.125.0.0 255.255.0.0 Google (Googlebot) Googlebot (74.125.0.0/16)
209.85.128.0 255.255.128.0 Google (Googlebot) Googlebot (209.85.128.0/17)
216.239.32.0 255.255.224.0 Google (Googlebot) Googlebot (216.239.32.0/19)
64.4.0.0 255.255.192.0 MSN/Live (MSNBot) MSNBot (64.4.0.0/18)
65.52.0.0 255.255.0.0 MSN/Live (MSNBot) MSNBot (65.52.0.0/16)
157.54.0.0 255.254.0.0 MSN/Live (MSNBot) MSNBot (157.54.0.0/15)
157.56.0.0 255.252.0.0 MSN/Live (MSNBot) MSNBot (157.56.0.0/14)
157.60.0.0 255.252.0.0 MSN/Live (MSNBot) MSNBot (157.60.0.0/14)
207.46.0.0 255.255.0.0 MSN/Live (MSNBot) MSNBot (207.46.0.0/16)
207.68.128.0 255.255.192.0 MSN/Live (MSNBot) MSNBot (207.68.128.0/18)
207.68.192.0 255.255.240.0 MSN/Live (MSNBot) MSNBot (207.68.128.0/20)
8.12.144.0 255.255.255.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (8.12.144.0/24)
66.196.64.0 255.255.192.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (66.196.64.0/18)
66.228.160.0 255.255.224.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (66.228.160.0/19)
67.195.0.0 255.255.0.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (67.195.0.0/16)
68.142.192.0 255.255.192.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (68.142.192.0/18)
72.30.0.0 255.255.0.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (72.30.0.0/16)
74.6.0.0 255.255.0.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (74.6.0.0/16)
202.160.176.0 255.255.192.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (202.160.176.0/18)
209.191.64.0 255.255.192.0 Yahoo (Yahoo! Slurp) Yahoo! Slurp (209.191.64.0/18)

Comments:

Hi there.

You have an error:

"157.60.0.0 255.252.0.0 MSN/Live (MSNBot) MSNBot (157.56.0.0/16)"

Should be:

"157.60.0.0 255.252.0.0 MSN/Live (MSNBot) MSNBot (157.60.0.0/16)"

Bye!

Posted by NotLim on korrik 29, 2012 at 07:18 MD BST #

Dear NotLim,
thanks for your correction of my typo!
I have updated the blog post to include your change.
Kind regards,
Stefan

Posted by sthieme on nëntor 19, 2012 at 01:13 PD GMT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

This blog covers various topics in the System Application Management Arena, i.e. the main Oracle tools are known as Oracle Real User Experience Insight and Oracle Application Testing Suite. It will from time to time take a look aside to see what integration offers with other tools of the Oracle Enterprise Manager Grid Control stack of tools.

Search

Categories
Archives
« prill 2014
DieHënMarMërEnjPreSht
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today