All marketers, whether high volume or not, want to purchase cheap, clean traffic.
That said, clean traffic that comes at a cheap price is a bit of a paradox. “You get what you pay for.” This is especially true when it comes to purchasing traffic for your site. Fact: Cheap traffic is not clean, and clean traffic is not cheap.
Conquering this obstacle can be a tricky task. So, how do you get the best of both worlds? In this article, we will go over the basics of Bot Traffic, three easy Bot Traps, and pro tips on how to deal with (and beat!) Bot Traffic.
Basics of Bot Traffic
Have you ever purchased cheap traffic, and noticed that a lot of that traffic was useless garbage? That’s because the majority of traffic that comes through cheap buys consists of bots.
So…What exactly is Bot Traffic? What is a Bot?
In short, a bot is a script which is written by publishers with shady intentions. These bots automatically visit that publisher’s site, sometimes up to several millions of times per hour. Each bot visit registers as an impression in your DSP, and they will charge you for each of those “impressions.” If you are running a CPC campaign, the bot can simulate a “click” on your ad, with the same result.
More of your money, with “fake” impressions to show for it. Bad bot!
You see, the DSP registers that a click occurred on your site, and will charge you for it. DSPs pay the publisher a percent of what you paid for, for the impression or click, whether a human user’s clicks or web bots’ clicks. On bot clicks, the DSP makes money, the publisher makes money… but you don’t.
How to deal with Bot Traffic
It goes without saying: this is not a good business model for DSPs.
The majority of DSPs out there already have a system in place to catch the more obvious bot traffic. But ultimately, it’s YOU who ends up paying for these fake impressions, so YOU need to know how to deal with it.
As previously mentioned, DSPs have created a system with filters to block bot traffic. However, the most effective bots are already finding their way through these filters. For you, that means putting a robot.txt file on your page will most likely be ineffective (AKA won’t do anything to stop these bots). That’s because robots ignore your robot.txt file(s). Malware robots and email address harvesters do this all the time to bypass good bots and access private information.
Malware Robots
Malware bots can gain control of a computer. A malware bot is like a spider, created especially to scan the web for vulnerable, unsecured server resources. If you have a folder of private information for work or passwords, make sure to keep it under locks. Set up your software so that a notification activates when your files are accessed, so you can see whether the access is from a trusted, real user or an advanced bot.
Email Address Harvesters
Typically used by spammers to send a spam trap, email harvesting consists of the process of obtaining email address lists, by use of various methods. Generally speaking, these email addresses are then used for bulk email or spam. While bots can use email harvesting to create spam or malware, a responsible user agent uses a bot trap to send relevant content to emails. The robots.txt file is publicly available, so anyone can see what section(s) of your server you don’t want robots to use.
DSPs take bot traffic seriously and usually offer refunds or credits to their customer, provided they report bot traffic and can prove it to be bot traffic. The best way to set up bot traps is through the use of an Ad Server. The Ad Server collects information on each impression and click that goes through your bot trap. You can use that information to prove to your DSP or traffic source that the quality of traffic is garbage, and you won’t be paying for it. You can also use the information you gather from your Ad Server to continuously improve your bot traps.
How to Set Up a Bot Trap
We will go over three ways to set up a bot trap below.
The basic set up for each trap method is the same. For each method, you will create at least two ad groups. The first ad group will include the ads you’re using in your campaign; we call this one your Default Ad Group. The second ad group, we’ll call this one Bot Traffic, will consist of (more than likely) just one ad image. This second ad group can have the same ad information you’re using in your Default Ad Group, but with a “dummy” redirect URL (that can go anywhere, Wikipedia, for example). These setups work even if you are using a link tracker placement to manage the URL redirects for your native, Facebook, or AdWords campaigns.
This simple set up will give you a brief overview/breakdown of your traffic impressions and clicks.
By clicking on the gear icon, you’ll open the Edit Ad Group page. This page lets you set up targeting conditions to ensure bot traffic goes to the Bot Traffic Ad Group.
#1: Device Type Targeting
This first method uses Uprival’s Targeting by Device Type.
To begin, edit the settings in your Bot Traffic Ad Group by selecting “Device Type” as the “Condition” and “Robot” as the “Device Type.”
Once you’ve done so, your targeting profile will look like this:
On your “Default Ad Group,” you’ll set up the opposite conditions by clicking the “Exclude Condition” box. When using targeting on ad groups or ads, it’s important to set up the opposite condition to make sure there is an ad targeting profile that looks like this:
With this targeting set up, your Default Ad Group (and the ads within that group) will be shown to all device types except “Robot.” Therefore, the Bot Traffic Ad Group will only show up on devices it identifies as “Bots.”
Although this is a simple and fairly easy setup, it poses the same problem as the robots.txt file we mentioned above. Since the device type field is self-identified, a bot would be required to tell the ad server that it is, in fact, a bot. Search engine bots can’t pass through using this targeting set up, other bots are programmed to “hide” or “change” the information that would self-identify them as a bot.
#2: IP Type Targeting
The next bot trap works by using Uprival’s IP Type Targeting.
Uprival can look up the IP address of the user and determine their IP Type. Some of the most common IP Types are as follows:
- Commercial (.com)
- Organization (.org)
- Residential
- Mobile
Because most real users would classify under the “Residential” or “Mobile” users group, we’ll use that for the Default Ad Group Targeting.
First, you’ll need to set up the Default Ad Group Targeting:
- Select Target IP Type as the condition for the Default Ad Group.
- Select Residential (Fixed Line) as the IP Type.
- Click “Add.”
- In a new Profile, select Target IP Type as the condition.
- Select “Mobile” as the IP Type.
- Click “Add.”
That should give you a profile that looks like this:
With Uprival, having multiple profiles in targeting treats the conditions as “OR” statements. So, our Default Ad Group will serve to Residential “OR” Mobile IP types.
From here, you will need to set up the Bot Traffic Targeting:
- Select Target IP Types as the condition for the Bot Traffic Ad Group.
- Check the “Exclude Condition” box.
- Select “Residential (Fixed Line)” as the IP Type.
- Click “Add.”
- Change the Profile to Profile 1, or whatever you’ve named your first profile.
- Select Target IP as the Condition.
- Make sure the “Exclude Condition” box is checked.
- Select “Mobile” as the IP Type
Your targeting profile should look like this:
UPRIVAL PRO TIP: When you add multiple conditions to the same profile, the program treats each condition as an “AND” statement.
Using this method, the Bot Traffic Ad Group will display to all IP types (EXCEPT for Residential and Mobile). This display is a reliable method because it is a lot harder, but not impossible, for bots to hide their IP type. Therefore, this method is not fool-proof, but it’s one of the most reliable bot traps available.
#3 Language Targeting
Another way to set up a bot trap is through Uprival’s Language Targeting.
Language Targeting is the newest targeting condition added by Uprival. It is meant to target ads based on the language(s) enabled on a user’s browser, but it also works as an effective bot trap.
Bots will have either no language or a language that differs from the local language in any given area.
Take a look at the numbers from an advertiser who’s currently running an ad campaign in France.
All the languages enabled on a browser get recorded as ISO 639-1 codes, with the preferred language listed first, and the rest in order of preference from left to right.
One thing that jumps out to me is the number of impressions with no browser language. Most browsers use an automatically set preferred language that doesn’t allow users to delete all of these languages (except Firefox). Now notice how many impressions come from browsers that do not have “fr” (France) anywhere in the language string. Though it’s wrong to assume all of these are bots, they are definitely outside of the target market for this campaign; this raises a red flag.
Okay, let’s begin! To start, you will need to set up the Default Ad Group Targeting:
- Select “Language” as the condition for the Default Ad Group.
- Select preferred language from the Language drop-down menu, and enter the IOS 639-1 code of the language you’d like to target.
- Click “Add.”
Next, set up the Bot Traffic Targeting
- Select “Language” as the condition for the Bot Traffic Ad Group.
- Check the “Exclude Condition” box
- Select preferred from the Language drop-down menu, and enter the IOS 639-1 code of the language you want to target.
- Click “Add”
This method works by sending all traffic (whose preferred browser language is NOT fr-FR for France) to your Bot Trap Ad Group.
Remember – Be prepared for greatness.
Sometimes, knowing how to deal with bot traffic is all it takes to turn a losing campaign into a winning one. If you’re using a traffic source with a lot of bot traffic, hit them up and get a refund. However, it’s always easier to do so when you have data to back up your claims.
The three traps we’ve set up are a great way to monitor and manage bot traffic more effectively. Get creative and combine the targeting techniques to increase your bot detection even more.
With an ad server like Uprival, bot traffic is as easy to detect and optimize as setting up a new traffic source.
Stay golden!
