Bot Detection

The Evolution of Bot Attacks

July, 21, 20206 min Read

A bot is an automated application that is programmed to do certain tasks that often mimic or replace human user's behavior. Bot technology has evolved over time which has allowed businesses to be more efficient but also leaves them vulnerable to attacks.

The dynamic of bot attacks

A wealth of information, personal data, and services are available online. On average, Google processes over 40,000 searches every second. If in need of information, one would first check the web before considering paying a visit to the nearest library. Social media and publicly available data has made it easier than ever to gather information about individuals. One no longer needs to go out to buy essentials or to manage finances. As more and more businesses move online, more opportunities are opened up to fraudsters. I’ve worked in defending sites against automated traffic and fraud for over 10 years and although the techniques used to defend businesses have significantly evolved, so have those used by attackers.

Back in the day, rate control, blacklists, and a handful of nicely crafted rules was pretty much all you needed to successfully defend against attacks. Surprisingly, these techniques can actually still work, but not for long. 

Let me explain: The sophistication of the attack is typically proportional to the defense in place, but also to the value of the data or goods collected as a result of a successful attack. If a site has no protection in place, an attacker has no incentive to deploy a more costly botnet spanning 1,000s of nodes in multiple continents with techniques designed to defeat bot detection. In this case, a simple curl script from a single machine will do. 

However, as soon as the site administrator starts to put some defenses in place, the game is on and things can evolve fairly rapidly! Having worked for a couple of vendors who offer bot and fraud detection products and having traded information with other vendors in the space, the way an attack evolves is always more or less the same.

Recommended Whitepaper: The Ultimate Guide to Bot Prevention

The six stages of a bot attack

Bot detection has advanced to become more modern and sophisticated against automated attacks. Fraudsters continuously study defense mechanisms that allow large-scale fraud attempts. The stages of a bot attack, launched by a fraudster typically follow these stages.

Stage 1: Introduction of a web security product. At this stage, the site administrator updates the workflow of their most critical endpoint and introduces a web security product. Existing bots attacking the site will be immediately detected with 100% accuracy simply because most botnets at that stage are scripts that are designed to do a particular task and tailored to follow a given workflow (for example create an account, login). The change introduced into the workflow by the web security product is not part of the default bot script.

Stage 2 - Scaling the botnet and impersonating browsers' signatures. Most basic bot defenses consist of mitigating with rate control, IP blacklist or defining rules that match the bot signature such as the user-agent. To defeat these basic techniques, bot operators will quickly scale their botnet to 1,000s of nodes. These nodes are hosted in cloud providers spanning several countries. The will also randomize their HTTP header signature (mostly the user-agent).

Stage 3 - Reverse engineering and replay. Of course, as soon as the bot operators realize that the attack continues to fail, they will re-evaluate the workflow. They will look for signs of web security protection and study it. That’s when the time comes for replays. The attacker will identify the type of information that is being collected by the product. They will then update the script to include a “good fingerprint” or try various combinations and randomly change some data points and evaluate the result. If the product uses some sort of persistent ID or cookies, the attacker will also attempt to harvest them from legit sessions and replay them from a botnet.

Stage 4 - Force the web security product to fail open. Failing to defeat the detection through replay, the most persistent attackers will try sending malformed data to see if it triggers some sort of exception. After all, bot operators are all developers and they know that software exceptions can cause the product to “fail safe”. In effect will disable the defense. They also know that detection engines don’t have infinite capacity and the fail-safe mechanism may trigger if overwhelmed with traffic. This last option can be a risky bet for the attacker though as they need to be careful not to overwhelm their target in the process, otherwise it’s game over.

Stage 5 - Upgrade the botnet to a headless browser. Products like selenium or headless chrome can be used to build a more intelligent botnet that can run JavaScript. It can be programmed to simulate human behavior all the way to key presses, mouse movements, and clicks. 

Stage 6 - Give up the botnet, move on to manual fraud. If all attack techniques against the bot detection feature of the product fail, certain types of attacks may still be cost-effective when done manually. More advanced techniques take advantage of IP intelligence, device intelligence, user contextual information, knowledge of sweatshop centers and behavioral detection methods are required in this case to successfully differentiate between legitimate and suspicious human traffic.

Recommended Case Study: Kicks Bots Out of Digital Streaming with Arkose Labs

The bottom line

The detection methods used at each stage work together to differentiate between automated, suspect human, and legitimate human traffic. Challenging all suspicious traffic with the right level of pressure will discourage the bot operator from continuing their attack, or make it cost-prohibitive. Each stage of the attack requires an increasing level of skill. Many will give up either because they do not have the expertise, or because the ROI is simply too low and they would rather target an easier, less protected, website. 

Advanced solutions to mitigate attacks

Arkose Labs’ innovative approach to step-up authentication provides graduated, risk-based friction that drains fraudsters’ resources – while allowing real customers a fun way to prove they are legitimate. Download this Arkose Enforce solution brief on how Arkose Enforce foils both automated and human-driven attacks.