Estimated Reading Time: 5 minutes
A Breakdown of Types of Bots, Traffic, and Web Crawlers
The size and magnitude of the Malware and Ad-fraud bot problem is immense and growing. And, as bots continue to proliferate, there’s important distinctions to point out between Malware bots and Ad-fraud bots. At zvelo, we’re always looking for ways our data can detect and mitigate bad bots and equip the smart minds who are working on solutions to stay one step ahead of the bad guys.
Let’s Start with the Bot Basics
The term “Bot” is a shorthand term for a web robot. Web bots are software applications that run automated tasks over the Internet. Many of these bots can be used for good, such as the web-crawler Google uses to categorize the web for search results. It’s called “Googlebot” and is one of the best examples of a “good” bot. The first web search engines were simply large scale bots that would crawl the web, fetching, analyzing and storing information from web servers so that humans could search the indexed results quickly.
Malware Bots & Botnets
Malicious bots are highly sophisticated, difficult to catch and are masters of disguise. Thousands of PCs infected with malware (also known as bot slaves) work in conjunction with a bot master to perform, among other tasks, online ad fraud by posing as actual human traffic.
The bot master decides which sites the slave accesses and which ads it views and clicks, so its actions appear to be random and to come from the computer of a “real person.”
Malware Bots Can Be Used To
- Harvest email addresses and other private or personal data
- Distribute viruses, worms, and other malware
- Generate Distributed Denial of Service attacks (DDoS)
- Control botnets (collections of bots) and zombie computers
Malicious botnets are collections of Internet-connected computers that connect to Command and Control (C&C) centers to download instructions for their malicious activities. From these downloaded instructions they can then determine where to direct the above activities, launching spam email campaigns or targeting their Distributed Denial of Service (DDoS) attacks. These botnets can operate in a client-server model, which can be relatively easy to disrupt, or a peer-to-peer model where the instructions for the latest attack are downloaded from the other nodes in the botnet. The peer-to-peer botnets are harder to disrupt, since removing a single node doesn’t disrupt the C&C dataflow.
Ad Fraud Bots
Ad Fraud bots are malicious bots with a specific end-goal in mind. It is worth noting that a bot can originate as a malicious bot and morph into carrying out Ad-fraud specific actions in a matter of hours or days, so whether in the InfoSec or Ad-Tech industry, it is important to be able to detect both types of bots.
Ad fraud happens when a bot attempts to imitate legitimate web traffic (acting like a real person visiting a website) and generate additional (but fraudulent) web page views (and therefore revenue) for the website publisher. The advertisers’ budgets are compromised, as their dollars are being wasted on ads being served to bots rather than humans. This results in the advertisers and the end users paying the price for this fraud, as well as being exposed to the risks associated with malicious and fraudulent bots. Recent estimates have put the cost of Ad fraud at 20-30% or more of the online advertising spending, or several billion dollars a quarter.
As you might imagine, advertisers and their suppliers are very interested in data about bots that are participating in Ad-Fraud. Online digital advertising is growing rapidly and is plagued by significant levels of Invalid Traffic (IVT) or non-human traffic (NHT) (these are Ad-Tech industry terms for web page views that aren’t from humans). Advertisers are increasingly insisting that their agencies and Demand Side Platform (DSP) partners utilize tools and data that reduce or eliminate IVT and NHT traffic to ensure that the online advertising they are purchasing is of high quality.
Some Tactics Deployed By Ad Fraud Bots
- Retargeting Fraud: This bot can mimic a human’s intentions, such as an interest in a specific brand of car. Ads targeted to a particular niche result in a higher CPM than untargeted ads. These bots deceive advertisers into believing they are receiving valuable, targeted traffic and clicks.
- Sophisticated Fraud: This type of bot travels around the web to visit websites, view ads and clicks using sophisticated algorithms. Think of it as a digital ghost that is always boosting the click-through numbers.
- Ad Fraud Botnets: These bots run quietly in the background of infected computers without making their presence known to the owner. Then, under the hacker’s remote control, the botnet—which can be rented through black-market Internet forums—is directed to visit certain websites. The most sophisticated bots are programmed to click from one website to another, watch videos for their duration, and even add items to an online shopping basket.
What Can Be Done About The Bots?
InfoSec professionals have their hands full with this growing challenge. Because machines that have become part of a botnet can deploy malware to their neighboring computers, threat intelligence about malicious botnets is vital to protecting corporate, as well as, online advertising assets. By keeping up to date on Command and Control (C&C) site lists, and blocking malicious bots on their edge devices such as Intrusion Prevention Systems (IPS) and Firewalls, they can disrupt the flow of instructions from the C&C hosts to any potential machines that have become part of a botnet.
While corporate InfoSec staff are concerned with their internal data being lost to a botnet controlled machine, they are less concerned with a 3rd party losing money on an ad campaign with many only motivated to deploy defense mechanisms because of the economic aspects of security failure. With greater awareness of the issue, more InfoSec professionals and advertisers will become involved in the fight against fraud. As financial impacts increase, they will take actions to minimize the bots creating fraud in their system. Companies that will be the ultimate winners in this space are proactively engaging third-party traffic validation technologies, and implementing mitigation tactics to combat online bot fraud. And that means brands will finally see their digital budgets put to efficient and effective use–advertising to humans, not bots.
