Understanding IP filtering: Common techniques and pitfalls

Using IP addresses to filter online traffic is a common practice. Yet, we increasingly see IP filtering being used as a blanket solution for some use-cases that just don’t make sense.

One recent client had a list of several hundred thousand IPs they were entering into a single text field in their programmatic platform! What a headache!

Especially within the world of advertising, IP filtering is a tedious process that has some serious limitations for companies who rely on it as the primary method for filtering out fraudulent behavior or identifying specific users.

Yes, we all know that IP filtering is effective and appropriate for some situations. For instance, IT professionals might want to block a specific subset of web traffic to their sites – maybe due to geographic location or by only allowing a certain allowlist of IP addresses.

Firewalls are built upon IP infrastructure and most website operators maintain firewalls to prevent access from known bad actors.

From a commerce standpoint, sites like Netflix (and many others) limit access to specific content based on IP addresses.

Even most spam filtering technologies rely on RBLs (Real-time Blackhole Lists), which are lists of IPs known for being sources of spam.

In order to avoid the common misconception that IP filtering is a foolproof solution for every problem, it’s important to understand some of the limitations before making it your preferred method of security and filtration.

What is an IP address exactly?

IP (or Internet Protocol) addresses are temporary identifiers assigned to each device on a network and help for routing and delivering network traffic.

They work similar to a physical mailing address and are included in communication much like a return address on an envelope.

When someone engages with you online, an IP provides an easy way to get back to them. Much like a physical mailing address, an IP address can be shared by many people and aren’t permanent.

There is a limit to the number of IP addresses that can be issued and the addresses in the most widely used format (called IPv4) have all been allocated.

To deal with this, a router at a home, school, office or public space will only broadcast one IP out publicly and it’s difficult to differentiate between individual devices on that network.

Continuing the analogy, devices (like people) can and do change addresses regularly.

The permanency of an IP address depends on the network issuing the identifier to the device. While large organizations might keep an address indefinitely, your home router might decided to expire an address after 24 hours if it goes unused.

An ISP might give a user a week or more before issuing a new address. In most cases, if users remain connected to the network they’ll keep the same address.

The pitfalls of IP blocking

The first major pitfall of IP blocking is that it’s really, really easy to intentionally thwart an IP blocker.

You don’t have to be an advanced hacker to take advantage of the many protection services available that hide information associated with IP addresses.

We mentioned earlier that Netflix provides different content to different countries (which is determined by IP addresses).

A variety of “unblocking” services are available for users who can mask their IP address in order to access a different inventory of TV shows and movies.

This is a very basic tool. Most users intentionally engaging in malicious or suspicious activity, especially if they are involved in ad fraud where content is geo-targeted, already block or change their IP address.

Secondly, much of today’s malicious traffic is routed through botnets (networks of malware infected computers that can be controlled centrally and without the computer owner's knowledge), so the IP addresses that appear to be causing the malicious behavior are often thousands of miles away from the actual bad actor.

The third pitfall is one of “bang for buck.” Usually, the most suspect users are the ones who frequently change their IP address, which makes maintaining accurate blocklists a very tedious act with a lot of administrative overhead.

The average bot lasts only six days and usually with a new bot comes a new IP.

This bot chase quickly turns into an endless game of whack-a-mole as old bots disappear and new, more sophisticated bots with new addresses appear in their place.

Lastly, since multiple people are likely to be using the same IP addresses at any given time, blocking entire IPs because of a single bad actor comes at the risk of inconveniencing real customers who operate under that IP.

In the world of advertising, blocklisting entire IPs can prevent campaigns from reaching actual target audiences and can lead to under delivery of campaigns.

Companies could be blocking many, many users who they don’t want to be excluding at all, simply because one person on that particular IP had an infected computer or another issue.

Some may say, “That’s not so bad. Block some of the good guys for the sake of blocking all the bad guys.”

There might be validity to that statement if it were true. The major concern is that IP blocklists are often missing the bad guys all together.

One study reported that 90% of all suspicious IP Addresses were not identified by blocklists.

Learn more about different types of traffic filtration methods by heading to Are You A Human’s blog.

***

Need data-related answers for your next marketing campaign or client partner? Contact The Data Hotline today. (What's The Data Hotline?) 

Photo: Monkey Business Images/Shutterstock