     [Blog](https://scrapfly.io/blog)   /  [blocking](https://scrapfly.io/blog/tag/blocking)   /  [How to Bypass Anti-Bot Protection in 2026: All 8 Major Vendors](https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection)   

# How to Bypass Anti-Bot Protection in 2026: All 8 Major Vendors

 by [Hisham Medhat](https://scrapfly.io/blog/author/hisham) Nov 18, 2026 26 min read [\#blocking](https://scrapfly.io/blog/tag/blocking) [\#python](https://scrapfly.io/blog/tag/python) [\#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) 

 [  ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection "Share on LinkedIn") [  ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection&text=How%20to%20Bypass%20Anti-Bot%20Protection%20in%202026%3A%20All%208%20Major%20Vendors "Share on X") [  ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection "Share on Facebook")    

 

 

Summarize this article with

 [  ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection) [  ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection) [  ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection) [  ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection) [  ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-anti-bot-protection) 



         

One of the biggest challenges in web scraping is anti-bot protection which can be run by many different vendors. However, we can reduce all of these vendors to a single fact - web scraper connections appear different compared to a web browser, and each vendor looks at a different set of these differences.

So which anti-bot system is blocking our scraper? In this article we'll take a look at 8 major anti-bot vendors - DataDome, PerimeterX, Kasada, Cloudflare, F5, Akamai, Incapsula and AWS WAF - and how to identify each one from its headers, cookies and scripts. We'll also take a look at how to patch the leaks each vendor is looking for and where a managed unblocker takes over.



## Key Takeaways

- The easiest way to tell which vendor is blocking us is by the cookies, headers and scripts it leaves behind - like `datadome` cookie or `CF-RAY` header. 429 or 403 status code on the other hand can be sent by any server, so it's not much help.
- Just plugging one leak is not enough either. Our `curl_cffi` request impersonated Chrome's TLS fingerprint, but the demo still returned 403. Matching the handshake alone wasn't enough.
- Kasada constantly changes its challenges and each website gets its own version of them, so DIY bypass that works today can break with the next update and needs fixing for every Kasada website separately.
- With `unblocker=True` and JavaScript rendering enabled Scrapfly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) got through `web-scraping.dev/antibot/easy` with a 200 response and the secret content. In Scrapeway's Sep 11-25, 2026 benchmark it scored 91% on Kasada and 100% on DataDome and PerimeterX!

**Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime.







## How to Identify Which Anti-Bot System You're Hitting

When it comes to bypassing anti-bot protection the first step is figuring out which vendor we're up against. Every anti-bot vendor has to leave some traces in the response, and the way these traces look can tell us exactly which vendor we're dealing with.

Anti-bot challenges are rather complicated, and we don't need to solve them to identify our problem, though some basics will help. For a deeper look at detection patterns see [our identification guide](https://scrapfly.io/blog/posts/how-to-know-what-anti-bot-website-uses), but for now let's take a quick look at the 4 places where vendors leave their signatures: headers, cookies, scripts and block pages.

### Response Headers and Status Codes

Headers are vital part of every HTTP response as they provide important meta information about the server. This varies from basic information like the server name, to custom vendor headers like Cloudflare's `CF-RAY`, DataDome's `X-DataDome-*` or Akamai's `akamai-grn` reference number.

Status code itself provides very little information about the vendor. A 429 (aka "too many requests") is defined in [RFC 6585](https://www.rfc-editor.org/rfc/rfc6585.html#section-4) and any server can send it, while a 403 can come from any firewall or even the website's own code. So, when all we have is a status code without any vendor headers or cookies we should keep digging.

### Cookie Name Patterns

Another data point that is commonly used in identifying vendors is cookie names. These names are unique to each vendor, so a `datadome` cookie points to DataDome and an `_abck` cookie points to Akamai.

One exception here is Cloudflare's `cf_clearance` cookie. We'd think it means we passed the challenge, but that's not always the case - according to [Cloudflare's docs](https://developers.cloudflare.com/cloudflare-challenges/challenge-types/javascript-detections/) this cookie can store a failed JavaScript detection result too.

### Script Tag and JS File Signatures

Javascript files the page loads are also a good indicator as these are consistent across all websites that use the same vendor.

### Block Page Visual Signatures

When we open the block page itself, most vendors brand it which is great for identification!

Kasada is the odd one out here as its block page is often blank, but we'll see how to spot it in the table below.

### Identification Reference Table

| Vendor | Cookies | JS Signature | Response Header | Block Page |
|---|---|---|---|---|
| Cloudflare | `cf_clearance`, `__cf_bm` | `/cdn-cgi/challenge-platform/` | `CF-RAY`, `server: cloudflare` | Branded interstitial + Ray ID |
| DataDome | `datadome` | `tags.js` | `X-DataDome-*` | Labeled CAPTCHA page |
| PerimeterX | `_px3`, `_pxvid`, `_pxhd` | `px.js` / `d.js` | None distinctive | HUMAN Security branding |
| Kasada | `KP_UIDz`, `x-kpsdk-ct` header | `ips.js` / `p.js` | None | Often minimal, unbranded |
| Akamai | `_abck` | akamai-bm-telemetry | `akamai-grn` | Reference number page |
| Incapsula | `incap_ses_*`, `visid_incap_*`, `reese84` | Imperva JS challenge | None distinctive | Imperva-branded challenge |
| AWS WAF | `aws-waf-token` | `/challenge.js` | `x-amzn-waf-action` (on challenge and CAPTCHA responses) | AWS-branded CAPTCHA |
| F5/Shape | `TS*`-prefix (e.g., `TS01xxxxxx`) | Shape AI JS | Shape/F5 error headers | Generic "Access Denied" |

We can check all of this with a short python script that prints out the response headers and cookies:

python

```python
import requests

url = "https://web-scraping.dev/antibot/easy"
response = requests.get(
    url,
    headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"}
)

print("Status:", response.status_code)
print("\nHeaders:")
for key, value in response.headers.items():
    print(f"  {key}: {value}")
print("\nCookies:")
for name, value in response.cookies.items():
    print(f"  {name}: {value[:60]}")
```



Our `web-scraping.dev/antibot/easy` demo gives us a 403 with `Server: uvicorn` and no vendor cookies - that's because it's a mock challenge and not a real anti-bot vendor.

[How Headers Are Used to Block Web Scrapers and How to Fix ItIntroduction to web scraping headers - what do they mean, how to configure them in web scrapers and how to avoid being blocked.](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-headers)



## How These Systems Detect Scrapers: Common Signals

When it comes to blocking web scrapers the main goal of every anti-bot vendor is difference spotting - is this client different from a general web browser? There are a lot of different signals that can answer this question and each vendor uses its own mix of them, so just copying a User-Agent string from a web browser is rarely enough to get through.

| Detection Layer | What's Checked | Which Vendors Use It |
|---|---|---|
| TLS / JA3 fingerprint | Cipher suite order, TLS extensions, GREASE values | Cloudflare, PerimeterX, Kasada, F5, DataDome |
| IP reputation | ASN, datacenter vs. residential, known proxy ranges | All eight vendors |
| HTTP header ordering | Header name order, casing, pseudo-header priority in HTTP/2 | Cloudflare, PerimeterX, Akamai, F5 |
| JS fingerprinting | Browser APIs, canvas, WebGL, hardware sensors | Kasada, DataDome, Akamai, PerimeterX |
| Behavior biometrics | Mouse path, scroll velocity, typing patterns | DataDome, PerimeterX, Kasada, Akamai |

Some of these signals can be checked instantly. So what can the vendor see before the page loads? Quite a lot actually - our IP address, TLS handshake (aka JA3 fingerprint) and HTTP headers are all sent with the very first request. Javascript fingerprinting is a bit different - it can only run once the page is loaded in our browser, but then it can extract a lot more details such as canvas rendering, hardware details, browser APIs and even our mouse movements.

### The Consistency Requirement

The important thing to note here is that all of these signals have to match. For example, if we send a Chrome User-Agent string over a Python TLS fingerprint it's an easy give away and any of these vendors can score it as a bot. In other words, if our [header configuration](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-headers) says we're Chrome then our [TLS fingerprint](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-tls) has to say Chrome too, and if we're using [browser fingerprint impersonation](https://scrapfly.io/blog/posts/bypass-proxy-detection-with-browser-fingerprint-impersonation) - that has to say the same thing as well!



## Which Bypass Techniques Still Work in 2026, and Where Does Each One Break?

Unfortunately, there's no single technique that works against every anti-bot system out there. Each technique below fixes one of the detection layers we've covered above and they are ordered roughly by cost - the cheapest ones first:

| Technique | What it fixes | Where it breaks |
|---|---|---|
| Realistic headers and request shape | Header order and User-Agent checks | The moment a TLS or JavaScript check runs |
| TLS and HTTP/2 fingerprint matching (`curl_cffi` with `impersonate="chrome"`) | The handshake check that flags Python clients | Any JavaScript challenge, because no JavaScript runs |
| Residential or mobile proxies | IP and ASN reputation | Behavior checks and challenges, and they cost per GB |
| Stealth or patched browser (nodriver, Camoufox, playwright-stealth) | Runs the JavaScript challenge | Per-site challenge rotation, and when the patch falls behind the browser release |
| Behavior simulation (mouse, scroll, timing) | Slider and press-and-hold challenges | Brittle and slow at volume |
| CAPTCHA solving | Clears the visible challenge | Doesn't fix the score that triggered it |

To see this in action let's start with the cheapest option. We'll send a request with a Chrome User-Agent header using `requests` (2.34.2) to the web-scraping.dev mock anti-bot page:

python

```python
import requests

url = "https://web-scraping.dev/antibot/easy"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
}
response = requests.get(url, headers=headers)
print(response.status_code)
print("You are a bot" in response.text)
```



Here, we get a 403 status code and the page tells us "You are a bot":

text

```text
403
True
```



Next, let's try fixing our TLS fingerprint. For this, we can use `curl_cffi` (0.17.0) which can impersonate Chrome's TLS and HTTP/2 handshake with a single `impersonate` argument:

python

```python
from curl_cffi import requests

url = "https://web-scraping.dev/antibot/easy"
# impersonate copies Chrome's TLS and HTTP/2 fingerprint
response = requests.get(url, impersonate="chrome")
print(response.status_code)
print("You are a bot" in response.text)
```



Unfortunately, we still get blocked:

text

```text
403
True
```



Changing the TLS fingerprint wasn't enough to get through. Chrome driven by browser-use also returned 403 with JavaScript enabled and disabled. These results don't tell us which check rejected the requests.

Each of these techniques is covered in more detail in our other articles. [How to Bypass Anti-Bot Protection When Web Scraping](https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection-when-web-scraping) covers five techniques end to end. [TLS fingerprinting](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-tls) and [curl-impersonate](https://scrapfly.io/blog/posts/curl-impersonate-scrape-chrome-firefox-tls-http2-fingerprint) cover the handshake, while [JavaScript fingerprinting](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-javascript) and our [nodriver tutorial](https://scrapfly.io/blog/posts/nodriver-tutorial) cover the browser rung.



Scrapfly

#### Need to bypass anti-bot protection?

Scrapfly's Unblocker handles Cloudflare, DataDome, and more — automatically.

[Try Free →](https://scrapfly.io/register)

## Bypass by Vendor

### DataDome

**Fingerprints**: `datadome` cookie, `tags.js` challenge script.

DataDome doesn't rely on static rules like most firewalls do. Every request is given a trust score by a machine learning model, and this model learns from all of the traffic DataDome sees (aka the ratio of bots to humans across all of it). So, there's no one rule we could work around - it's the whole request that gets scored. When a request needs human verification DataDome shows a slider CAPTCHA, which records our mouse path, timing and pressure while we solve it.

This means that a website with strict DataDome settings can require pretty much everything we've covered so far. So where do we start? Well, the handshake is the easiest part - our TLS and HTTP/2 fingerprint should look like the browser named in our User-Agent. Our IP address is next. Residential or mobile proxies are a must here as a bad IP can pull the whole score down. For the `tags.js` checks we'll need a stealth browser and when the slider shows up - some behavior simulation too!

Also, DataDome's model is retrained all the time. This means even a setup that worked fine last week can suddenly start getting challenged, and we didn't change a thing!

Scrapfly's Unblocker scored 100% on DataDome in Scrapeway's benchmark (Sep 11-25, 2026), as listed on the [DataDome bypass page](https://scrapfly.io/bypass/datadome).

For a complete DataDome bypass tutorial: [How to Bypass DataDome Anti-Scraping](https://scrapfly.io/blog/posts/how-to-bypass-datadome-anti-scraping).

### PerimeterX / HUMAN Security

**Fingerprints**: `_px3` clearance cookie, `px.js` challenge script, HUMAN Security branding on block pages.

PerimeterX works in two steps. First, before the page even loads, the backend scores our IP address and ASN (datacenter ranges included). Whether a low score gets blocked, challenged or allowed is up to the site owner. The second step is the `px.js` script which sends our behavior data to the `/api/v2/collector` endpoint. There's also the "Press &amp; Hold" challenge which records how long and how hard we press the button.

So which techniques do we need here? As the IP score comes first, residential or mobile proxies are the place to start. Then, we need a stealth browser that actually runs `px.js` - replaying the collector requests without a real browser can be detected. Finally, the "Press &amp; Hold" challenge calls for behavior simulation. Of course, all of this depends on how the website configured PerimeterX.

The important thing to note here is that every PerimeterX customer gets their own machine learning model. So, a setup that works on one website needs to be tested again on the next one and again whenever that website's model changes.

Scrapfly scored 100% on PerimeterX in Scrapeway's Sep 11-25, 2026 benchmark (see the [PerimeterX bypass page](https://scrapfly.io/bypass/perimeterx)).

For a complete tutorial including session management: [How to Bypass PerimeterX When Web Scraping](https://scrapfly.io/blog/posts/how-to-bypass-perimeterx-human-anti-scraping).

### Kasada

**Fingerprints**: `ips.js` / `p.js` challenge script, `KP_UIDz` cookie, `x-kpsdk-ct` header, encrypted rotating payloads.

Kasada works quite differently compared to other anti-bot services. Instead of passively reading our fingerprint it actively interrogates (aka probes) the javascript environment and collects behavior data like mouse path, scroll velocity and hardware sensor readings. It also hides its challenge code behind obfuscated eval chains with anti-debugging tricks - if we try to deobfuscate or reverse it, Kasada will notice. And to make it even harder, challenges rotate with each deployment!

Note that Kasada's block page is often blank so the best way to spot it is the `KP_UIDz` cookie or `x-kpsdk-*` headers.

To get past Kasada we need to go all the way up our technique table. This is where a stealth or patched browser comes in as the challenge has to run in a real environment. We'll also need behavior simulation for the mouse, scroll and sensor checks and residential proxies, as IP reputation still counts.

Unfortunately, keeping all of this working is a lot of effort. A DIY setup can break with any challenge update, and because challenges rotate per site we'd have to fix it again for every Kasada website we scrape!

Kasada has the lowest of the six published figures - Scrapfly scored 91% in Scrapeway's Sep 11-25, 2026 benchmark (see the [Kasada bypass page](https://scrapfly.io/bypass/kasada)).

For Kasada-specific details: [How to Bypass Kasada Anti-Bot When Web Scraping](https://scrapfly.io/blog/posts/how-to-bypass-kasada-anti-scraping-waf).

### Cloudflare Bot Management

**Fingerprints**: `cf_clearance` challenge-result cookie, `__cf_bm` bot management score cookie, `CF-RAY` response header, scripts under `/cdn-cgi/challenge-platform/`.

Cloudflare is a bit confusing at first as it has two products that often get mixed up - Bot Management and Turnstile. Turnstile is a challenge widget that can run non-interactive JavaScript checks without showing a CAPTCHA. Bot Management is the bigger one of the two. This is the one that scores every request we make. It also runs something called JavaScript Detections invisibly in the background (Cloudflare treats these as their own challenge type, different from Challenge Pages and Turnstile).

How does that score work? Cloudflare's [bot score](https://developers.cloudflare.com/bots/concepts/bot-score/) mixes heuristics, machine learning on request features and the JavaScript Detections we just mentioned. So what happens when our score is low? That's for the website owner to decide - we might get blocked, challenged or even let through. Some owners also add rules for specific ASNs and that's how datacenter IPs get filtered out.

When it comes to TLS, Cloudflare compares our TLS (JA3) fingerprint and HTTP/2 SETTINGS frame with the User-Agent and Sec-CH-UA headers we send. So, our Chrome User-Agent has to come with a Chrome TLS handshake as well. Finally, there's the proof-of-work challenge page which can only be solved by a real browser.

So which techniques do we need? That depends on the website's rules. For handshake checks that's TLS and HTTP/2 fingerprint matching, for ASN rules residential proxies and for JavaScript Detections or the challenge page a real browser. Unfortunately, browsers are also the hardest to keep up to date - once our patched browser falls behind Chrome releases it doesn't look like the browser it claims to be anymore.

Scrapfly scored 97% on Cloudflare in Scrapeway's Sep 11-25, 2026 benchmark, listed on the [Cloudflare bypass page](https://scrapfly.io/bypass/cloudflare).

For the full Cloudflare bypass approach: [How to Bypass Cloudflare Anti-Scraping](https://scrapfly.io/blog/posts/how-to-bypass-cloudflare-anti-scraping). If you're using FlareSolverr, see our [FlareSolverr guide](https://scrapfly.io/blog/posts/flaresolverr-guide).

### F5 BIG-IP / Shape Security

**Fingerprints**: `TS*`-prefixed cookies (e.g., `TS01a2b3c4`), HTTP/2 SETTINGS frame, Shape AI JS challenge.

F5 BIG-IP is a bit different from the rest as it runs on the network level instead of the application layer - right in the traffic path. The bot protection itself comes from Shape Security, which F5 bought in 2020. So, by the time our request reaches the website F5 already knows a lot about our connection. It inspects our TLS and HTTP/2 fingerprints almost as thoroughly as PerimeterX and checks our IP address reputation and ASN. However, F5 itself doesn't decide what happens to poorly scoring connections - this is up to each website's policy which can either block or challenge them. That challenge is Shape AI javascript which, much like Kasada, actively pokes around our browser environment!

So to get through F5 our scraper needs a TLS and HTTP/2 fingerprint that matches a real browser, residential proxies and a stealth browser for the times Shape challenge shows up.

When it comes to proxies we recommend taking a look at our [residential proxy selection](https://scrapfly.io/blog/posts/introduction-to-proxies-in-web-scraping) guide first. Ideally, we should avoid suspicious ASN ranges altogether (see our [IP-based detection](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-ip-addresses) article) and spread our connections through many different IPs so they don't form a pattern - which is what [rotating residential proxies](https://scrapfly.io/blog/posts/how-to-rotate-proxies-in-web-scraping) are for.

On top of that, Shape's javascript challenge keeps evolving with each update, and every new version can break our DIY browser setup all over again.

F5 is one of the two vendors without a published figure - Scrapfly's [Scrapfly F5 bypass page](https://scrapfly.io/bypass/f5) only lists current approach notes.

For the full F5 walkthrough: [How to Bypass F5 Bot Defense When Web Scraping](https://scrapfly.io/blog/posts/how-to-bypass-f5-when-web-scraping).

### Akamai Bot Manager

**Fingerprints**: `_abck` cookie, akamai-bm-telemetry scripts, `akamai-grn` reference header on block pages.

Akamai's bot protection revolves around one cookie - `_abck`. Akamai's javascript sensor script watches how we behave, puts this telemetry into the cookie and then checks it server-side on every single request.

What if we copy a valid `_abck` from our browser into our scraper? Not really an option - Akamai expects the telemetry behind the cookie as well and that only comes from the sensor script running in a real browser.

This is why for Akamai we need a stealth browser that runs the sensor script, behavior simulation (as `_abck` stores behavior data) and residential proxies. Note that whenever Akamai updates its sensor script our browser setup needs another round of testing to make sure Akamai still accepts its telemetry.

Scrapfly scored 98% on Akamai in Scrapeway's Sep 11-25, 2026 benchmark (see the [Akamai bypass page](https://scrapfly.io/bypass/akamai)).

For full implementation details: [our Akamai bypass guide](https://scrapfly.io/blog/posts/how-to-bypass-akamai-anti-scraping).

### Incapsula / Imperva

**Fingerprints**: `reese84` JS challenge, `incap_ses_*` session cookie, `visid_incap_*` visitor ID.

Incapsula protects websites with its `reese84` challenge which is a mix of proof-of-work and browser fingerprinting. Until we solve it, we won't see any page content. Can we copy the solution from another session? Nope, it has to be solved by us! Even then the cookies we get only last for a while - once they expire Incapsula simply challenges us again.

So, for strict Incapsula websites we need a stealth browser that solves `reese84` and passes its fingerprint checks, plus residential proxies. Unfortunately, this happens on every new session - and whenever Incapsula tweaks `reese84`, our DIY setup has to be fixed as well!

Scrapfly scored 98% on Incapsula in Scrapeway's Sep 11-25, 2026 benchmark (see the [Incapsula bypass page](https://scrapfly.io/bypass/incapsula)).

For full implementation details: [our Incapsula / Imperva bypass guide](https://scrapfly.io/blog/posts/how-to-bypass-imperva-incapsula-anti-scraping).

### AWS WAF Bot Control

**Fingerprints**: `aws-waf-token` cookie, `/challenge.js` telemetry endpoint, AWS-branded CAPTCHA page.

AWS WAF works a bit differently as it's a full Layer 7 firewall rather than just an anti-bot tool. Bot Control is only one of its features and it sits next to DDoS protection, rate limiting and custom rules. Bot Control itself uses machine learning to classify requests, however it's up to each website operator to choose which rules are enabled. Meaning, we can come across one AWS WAF website that only limits requests per IP and another one that does full behavior fingerprinting!

The `aws-waf-token` cookie is the main thing to look out for here. Whenever our request matches one of the Challenge or CAPTCHA rules AWS checks for this token, and if it's missing (or expired) instead of the page we'll receive a challenge with status code 202 or a CAPTCHA with 405.

So, how do we get past AWS WAF? Well, that depends on which rules the website has enabled. Basic rate limits and IP rules can be handled with residential proxies, while challenges need a real browser - which conveniently gets us the token as well. If we run into the AWS-branded CAPTCHA though, we'll need CAPTCHA solving on top. As every website is set up differently, we should test our setup on each AWS WAF target separately.

AWS WAF is the other vendor without a published figure - Scrapfly's [Scrapfly AWS WAF bypass page](https://scrapfly.io/bypass/aws-waf) only lists current approach notes.

For the full AWS WAF walkthrough: [How to Bypass AWS WAF Bot Control When Web Scraping](https://scrapfly.io/blog/posts/how-to-bypass-aws-waf-when-web-scraping).

[How to Bypass Anti-Bot Protection When Web ScrapingLearn how anti-bot systems detect scrapers and 5 universal bypass techniques including proxy rotation, fingerprinting, and fortified headless browsers.](https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection-when-web-scraping)



## Cross-Vendor Comparison: Choosing Your Strategy

So, how do all of these vendors compare? Let's put all eight of them next to each other:

| Vendor | Detection model | Techniques its signals can call for | Managed-service benchmark |
|---|---|---|---|
| DataDome | ML trust score per request | TLS matching, residential proxies, stealth browser, behavior simulation | [100%](https://scrapfly.io/bypass/datadome) |
| PerimeterX | IP scoring, then JS challenge with telemetry | Residential proxies, stealth browser, behavior simulation | [100%](https://scrapfly.io/bypass/perimeterx) |
| Kasada | Active environment interrogation | Stealth browser, behavior simulation, residential proxies | [91%](https://scrapfly.io/bypass/kasada) |
| Cloudflare | Bot score from heuristics, ML on request features and JavaScript Detections | TLS matching, residential proxies, real browser | [97%](https://scrapfly.io/bypass/cloudflare) |
| F5/Shape | Network-level connection fingerprinting plus JS probing | TLS matching, residential proxies, stealth browser | Not published |
| Akamai | Server-side `_abck` sensor validation | Stealth browser, behavior simulation, residential proxies | [98%](https://scrapfly.io/bypass/akamai) |
| Incapsula | `reese84` proof-of-work plus fingerprinting | Stealth browser, residential proxies | [98%](https://scrapfly.io/bypass/incapsula) |
| AWS WAF | Configurable managed rule group | Residential proxies, real browser, CAPTCHA solving | Not published |

Note that the techniques column is based on each vendor's documented signals and on how the website configured it, so it's not a tested pass/fail matrix. The benchmark column on the other hand comes from Scrapeway's Sep 11-25, 2026 benchmark of Scrapfly's managed service and not from a DIY setup.

### DIY vs. Scraping API: When Each Makes Sense

So, should we build our own bypass or use a scraping API? DIY can work well if we're only targeting one vendor at low volume and have the engineering time to keep the bypass working. Think internal tools or personal projects - if our scraper fails every now and then, it's not the end of the world!

Once we're dealing with several vendors though, scraping at production scale or working with a small team, a scraping API starts to make a lot more sense. Why? Because a bypass is never really finished! Every vendor update can break it and we'd have to rework it again - a cost that's easy to forget when comparing DIY with an API.

Current Scrapfly prices live on the [pricing page](https://scrapfly.io/pricing).

[How TLS Fingerprint is Used to Block Web Scrapers?TLS fingeprinting is a popular way to identify web scrapers that not many developers are aware of. What is it and how can we fortify our scrapers to avoid being detected?](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-tls)



## Using Scrapfly Across All Eight Anti-Bot Systems



Both of our clients above stopped at the 403. As you can see, bypassing anti-bot protection is an enormous subject - and every vendor needs a slightly different approach! This is why Scrapfly's Unblocker exists. With one `unblocker=True` parameter it can handle all eight vendors we've covered here, plus 25+ anti-bot systems in total (the older `asp=True` parameter still works as well). All we have to do is turn it on - Unblocker figures out which vendor is protecting the page and handles the bypass itself.

Under the hood it deals with everything we've talked about in this article. Our TLS (JA3) and HTTP/2 fingerprints look like a real browser's, requests go through a proxy pool in 190+ countries and get switched to residential IPs when a target needs it, and the browser fingerprint stays coherent and as close to a real user's as possible. On top of that, CAPTCHA challenges are solved automatically and cookies and clearance tokens are tracked between our requests.

Let's give it a shot with our demo page! For this we'll need the Python SDK, `parsel` for parsing and a Scrapfly API key in the `SCRAPFLY_KEY` environment variable:

shell

```shell
$ pip install scrapfly-sdk parsel
```



python

```python
import os
from scrapfly import ScrapflyClient, ScrapeConfig

client = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"])
result = client.scrape(ScrapeConfig(
    url="https://web-scraping.dev/antibot/easy",
    unblocker=True,  # requests and curl_cffi got a 403 here
    render_js=True,
    country="US",
))

print("status:", result.status_code)
# the label and the value sit in separate elements, so read their parent
label = result.selector.xpath('//*[contains(text(), "Secret content")]')[0]
print(label.xpath("normalize-space(..)").get())
```



Here, we enable `unblocker` together with javascript rendering and look for the secret token that's hidden behind the bot check. Running it with scrapfly-sdk 0.12.0 logs a warning that `session_sticky_proxy` is ignored and then prints:

text

```text
status: 200
Secret content: You are a real browser! Token: abc123-secret
```



That's the same page that blocked both `requests` and `curl_cffi` earlier! Note that Python isn't the only option either - Scrapfly has TypeScript, Go and Rust SDKs with the same scrape options.

Vendor-specific bypass pages with current approach notes:

- [Cloudflare bypass](https://scrapfly.io/bypass/cloudflare)
- [Akamai bypass](https://scrapfly.io/bypass/akamai)
- [DataDome bypass](https://scrapfly.io/bypass/datadome)
- [Incapsula bypass](https://scrapfly.io/bypass/incapsula)
- [AWS WAF bypass](https://scrapfly.io/bypass/aws-waf)
- [PerimeterX bypass](https://scrapfly.io/bypass/perimeterx)
- [F5 bypass](https://scrapfly.io/bypass/f5)
- [Kasada bypass](https://scrapfly.io/bypass/kasada)

For full Unblocker documentation: [Scrapfly Unblocker docs](https://scrapfly.io/docs/scrape-api/unblocker).



### Web Scraping API

Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in.



[Try Web Scraping API](https://scrapfly.io/docs/scrape-api/getting-started)



## FAQ

Is bypassing anti-bot protection legal?

Yes. Bypassing anti-bot protection to reach publicly available data is legal in most jurisdictions as long as we're not extracting anything personal or private. That being said, attention should be paid when bypassing logins, scraping private data or breaking a website's terms in a way that causes measurable harm, as this is where it gets legally risky.







Can a bot bypass CAPTCHA automatically?

Yes. Most scraping APIs and headless browser setups solve CAPTCHAs on their own by combining browser emulation with a CAPTCHA solving service. That being said, CAPTCHAs only appear when our trust score is low, so the best approach is to fortify our scraper's fingerprint and avoid them altogether.







What's the hardest anti-bot system to bypass?

That would be Kasada. Out of Scrapfly's six published figures Kasada's is the lowest at 91% (Scrapeway's benchmark, Sep 11-25, 2026). On top of that, every Kasada website gets its own rotating challenges which makes DIY bypasses a pain to maintain.







How is Kasada different from a WAF?

WAF stands for web application firewall and it's a much broader Layer 7 tool. It can protect a website from DDoS attacks, SQL injection and too many requests (rate limiting) as well as from bots. Kasada on the other hand only cares about bots - all it does is interrogate the browser environment and analyze how we behave.







How do I know which anti-bot a site is using?

The easiest way is to take the headers and cookies snippet from our [identification section](#how-to-identify-which-anti-bot-system-youre-hitting) and run it against the website we want to scrape. What it prints can then be compared with the table above. So, a `CF-RAY` header would point to Cloudflare while an `_abck` cookie would point to Akamai. However, if all we get is a 429 or a 403 without any vendor headers or cookies, we'll need to keep digging.







How does PerimeterX work?

PerimeterX checks us in two steps. First, our IP address is scored on the backend before the page even loads. The second step is the `px.js` script which runs in our browser and sends our behavior data to PerimeterX's collector API. Pass both of these and we're given the `_px3` clearance cookie!









## Summary

In this article, we've taken a look at how to bypass anti-bot protection from 8 major vendors - DataDome, PerimeterX, Kasada, Cloudflare, F5, Akamai, Incapsula and AWS WAF. We started off by identifying which vendor is blocking us using headers, cookies and script paths, as the status code on its own (like a bare 429 or an unsigned 403) isn't enough to name the vendor.

Then, we've taken a look at which signals these vendors use and which bypass techniques can fix them. For example, we've tried fixing our TLS fingerprint with `curl_cffi` on our demo page, but it still got the same 403 as plain `requests`! That's why we should always start small (with the cheapest technique the vendor's signals call for) and see if it's enough before moving on to the next one. Just remember that each technique we add has to be maintained every time the vendor updates its challenge.

All that being said, bypassing anti-bot protection is a constantly moving target, so it's a perfect place for an abstraction layer! Scrapfly's Unblocker takes care of all eight vendors with a single `unblocker=True` parameter - it's the same call that got us our demo page's secret token. According to Scrapeway's Sep 11-25, 2026 benchmark it scores between 91% (Kasada) and 100% (DataDome and PerimeterX), so give it a try!



Legal Disclaimer and Precautions

This tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect:

- Do not scrape at rates that could damage the website.
- Do not scrape data that's not available publicly.
- Do not store PII of EU citizens protected by GDPR.
- Do not repurpose *entire* public datasets which can be illegal in some countries.

Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer.

 

   [  Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents















 

  Table of Contents

- [Key Takeaways](#key-takeaways)
- [How to Identify Which Anti-Bot System You're Hitting](#how-to-identify-which-anti-bot-system-you-re-hitting)
- [Response Headers and Status Codes](#response-headers-and-status-codes)
- [Cookie Name Patterns](#cookie-name-patterns)
- [Script Tag and JS File Signatures](#script-tag-and-js-file-signatures)
- [Block Page Visual Signatures](#block-page-visual-signatures)
- [Identification Reference Table](#identification-reference-table)
- [How These Systems Detect Scrapers: Common Signals](#how-these-systems-detect-scrapers-common-signals)
- [The Consistency Requirement](#the-consistency-requirement)
- [Which Bypass Techniques Still Work in 2026, and Where Does Each One Break?](#which-bypass-techniques-still-work-in-2026-and-where-does-each-one-break)
- [Bypass by Vendor](#bypass-by-vendor)
- [DataDome](#datadome)
- [PerimeterX / HUMAN Security](#perimeterx-human-security)
- [Kasada](#kasada)
- [Cloudflare Bot Management](#cloudflare-bot-management)
- [F5 BIG-IP / Shape Security](#f5-big-ip-shape-security)
- [Akamai Bot Manager](#akamai-bot-manager)
- [Incapsula / Imperva](#incapsula-imperva)
- [AWS WAF Bot Control](#aws-waf-bot-control)
- [Cross-Vendor Comparison: Choosing Your Strategy](#cross-vendor-comparison-choosing-your-strategy)
- [DIY vs. Scraping API: When Each Makes Sense](#diy-vs-scraping-api-when-each-makes-sense)
- [Using Scrapfly Across All Eight Anti-Bot Systems](#using-scrapfly-across-all-eight-anti-bot-systems)
- [FAQ](#faq)
- [Summary](#summary)
 
    Join the Newsletter  Get monthly web scraping insights 

 

  



Scale Your Web Scraping

Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits.

  No credit card required  1,000 free API credits  Anti-bot bypass included 

 [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) 

 Not ready? [Get our newsletter](#newsletter-modal) instead. 

 

 

## Related Articles

 [     

 blocking nodejs 

### Web Scraping With Node-Unblocker

Tutorial on using Node-Unblocker - a nodejs library - to avoid blocking while web scraping and using it to optimize web ...

 

 ](https://scrapfly.io/blog/posts/web-scraping-with-node-unblocker) [     

 blocking proxies 

### How to Choose a Web Unblocker for Web Scraping (and When a Proxy Is Enough)

When does a proxy stop being enough for web scraping? A 2026 guide to picking proxies, web unblockers, or scraping brows...

 

 ](https://scrapfly.io/blog/posts/how-to-choose-the-best-proxy-unblocker) [     

 python crawling 

### How to Find All URLs on a Domain

A hands-on guide to listing the URLs of a domain from robots.txt, XML sitemaps and a scoped Python crawl, with what each...

 

 ](https://scrapfly.io/blog/posts/how-to-find-all-urls-on-a-domain) 

  



   



 Bypass anti-bot protection automatically, **1,000 free credits** [Start Free](https://scrapfly.io/register)