[Blog](https://scrapfly.io/blog) / [blocking](https://scrapfly.io/blog/tag/blocking) / [How to Bypass Imperva Incapsula when Web Scraping in 2026](https://scrapfly.io/blog/posts/how-to-bypass-imperva-incapsula-anti-scraping) # How to Bypass Imperva Incapsula when Web Scraping in 2026 by [Bernardas Alisauskas](https://scrapfly.io/blog/author/bernardas) Oct 23, 2026 45 min read [\#blocking](https://scrapfly.io/blog/tag/blocking) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping&text=How%20to%20Bypass%20Imperva%20Incapsula%20when%20Web%20Scraping%20in%202026 "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-bypass-imperva-incapsula-anti-scraping) Imperva (aka Incapsula) is a popular WAF service used by many websites like giffgaff, Coles and pokemoncenter.com. This service is used to block bots, such as web scrapers from accessing the website. So, to scrape public data from these websites, the scrapers need to bypass Imperva Incapsula bot protection. In this article, we'll explain how to bypass Imperva's anti-scraping protection. We'll start by taking a quick look at what is Imperva, how to identify it and how does it identify web scrapers. Then, we'll take a look at existing techniques and tools for bypassing Imperva bot protection, such as curl\_cffi, Nodriver, SeleniumBase and Camoufox. *we checked every version and date in this guide between October 9 and 13, 2026, and ran the four open-source examples against web-scraping.dev* [How to Bypass Anti-Bot Protection When Web ScrapingLearn how anti-bot systems detect scrapers and 5 universal bypass techniques including proxy rotation, fingerprinting, and fortified headless browsers.](https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection-when-web-scraping) ## Key Takeaways Here's what we found while testing Imperva websites for this guide: - giffgaff.com sent us its "Request unsuccessful" block page with a 200, so `raise_for_status()` never complained and we had to check the body. - The `X-Iinfo` header and `incap_ses` cookie came back from imperva.com's real homepage too. - On pokemoncenter.com, a Chrome User-Agent alone got the "Pardon Our Interruption" page, while a full set of Chrome headers got plain `requests` the real homepage. - `curl_cffi` had the same cipher part as Chrome in its JA4 fingerprint, but it can't run the script behind the `reese84` cookie. - Headless Chrome got that cookie and was still blocked, with `HeadlessChrome/154.0.0.0` in its User-Agent, so we run browsers headful on residential proxies. - A blocked IP can recover. Thalesgroup.com let ours back in the next morning. - For scraping at scale, Scrapfly's [Imperva bypass API](https://scrapfly.io/bypass/incapsula) handles this for us. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## What is Imperva (aka Incapsula)? Imperva (previously known as Incapsula) is a WAF service suite that is used to protect websites from unwanted connections. It has legitimate uses in the context of web scraping, which is blocking web scrapers from accessing public data. In this guide, we'll only cover Imperva's bot protection, which it sells as Advanced Bot Protection. Getting past its WAF rules against attacks like SQL injection or XSS isn't part of web scraping, so we'll leave that out. Imperva/Incapsula is one of the first WAF services to be used by websites to block web scraping attempts and is generally well understood by the web scraping community. Imperva's cookies and block pages still use the Incapsula name. If you're not sure which anti-bot service a website uses, our guide on [identifying anti-bot services](https://scrapfly.io/blog/posts/how-to-know-what-anti-bot-website-uses) covers the general approach. So, let's explain how to identify Imperva and how it's identifying web scrapers. ## How Do You Know a Site Uses Imperva? The quickest way to find out is to request a page and look at what comes back. Here's what we got from three Imperva websites when we requested their homepages with curl and a Chrome User-Agent. | Website | Status | Body size | What the body contains | Imperva cookies | Imperva headers | |---|---|---|---|---|---| | giffgaff.com | 200 | About 1 KB | "Request unsuccessful. Incapsula incident ID" and a number | `visid_incap_2118422`, `incap_ses_975_2118422` | `X-Iinfo` | | coles.com.au | 200 | 212 bytes | An empty body and one script from `/_Incapsula_Resource` | `visid_incap_2800108`, `incap_ses_978_2800108` | `X-Iinfo` | | imperva.com | 200 | About 249 KB | The real homepage | `nlbi_2439_3105325`, `visid_incap_2439`, `incap_ses_975_2439` | `X-Iinfo`, `X-CDN: Imperva` | As we can see, all three responses came with a 200 status code, Imperva's `visid_incap` and `incap_ses` cookies and an `X-Iinfo` header. Yet only imperva.com returned its actual homepage. The giffgaff.com response is Imperva's block page, which is about 1 KB. Here's its HTML, trimmed a bit. html ```html ... ``` The only text in it is "Request unsuccessful" and an incident ID, and both the script and the iframe load from `/_Incapsula_Resource` on giffgaff's own domain. The Coles response is even smaller. It's a 212-byte page with an empty body, and all it does is load a script from `/_Incapsula_Resource`, which needs a browser to run. On the other hand, imperva.com returned its full homepage with the same cookies and `X-Iinfo` header, plus an `nlbi` cookie and an `X-CDN: Imperva` header. So, Imperva's headers and cookies only tell us that Imperva is there, and we need to look at the body to know whether we got blocked. Here's the full list of Incapsula block fragments. - `Powered By Incapsula` or `Powered by Imperva` text snippet in HTML. - `Incapsula incident ID` keyword in HTML. - `_Incapsula_Resource` keyword in HTML. - `subject=WAF Block Page` keyword in HTML. - `visid_incap` value in request headers. - `X-Iinfo` response header, and sometimes an `X-CDN` header naming Incapsula or Imperva. - `Set-Cookie` header has cookie field `incap_ses` and `visid_incap`, often with `nlbi`. Other Imperva blocks come with a 403 or another 4xx or 5xx status code, and some websites show a security check page with a [reCAPTCHA checkbox](https://scrapfly.io/blog/posts/how-to-bypass-captcha-web-scraping) instead. That security check page says Incapsula "will remember you and will not show this page again". That memory is kept in Imperva's cookies, so a client that drops them sees the check again. These errors are mostly encountered on the first request to the website. However, it can block the requests at any point during the scrape process as it constantly tracks connections. To skip the manual checks, the [Scrapfly Antibot Detector](https://scrapfly.io/products/antibot-detector) tells us which anti-bot service a website uses. And if a 403 shows up without any of these Imperva traces, our [403 guide](https://scrapfly.io/blog/posts/403-forbidden-web-scraping) covers the other usual causes. ### Why Can an Imperva Block Return HTTP 200? Blocked pages can appear in status code 200 to confuse web scrapers, which is what we got from giffgaff.com and coles.com.au. This is the worst kind of block for a scraper. When I requested giffgaff.com with Python's `requests`, `response.raise_for_status()` didn't raise anything for the block page, as its status code was 200. So, **we check the response body on every request**, not only the status code. To test the body check, I ran `is_imperva_block()` from the `curl_cffi` example below on the three responses from the table, with each website's name as the expected text. It returned `True` for giffgaff.com and coles.com.au, and `False` for imperva.com, whose homepage has the word "Imperva" in it along with an `_Incapsula_Resource` script. ### What Do X-Iinfo and the Incapsula Incident ID Tell You? Every Imperva block page represents a unique blocking incident, and it gets its own incident ID. Hence, knowing how to get past Imperva relies on identifying and understanding the error encountered. The `X-Iinfo` header came with every Imperva response we got, normal pages included. Here are three of them from October 13, from a giffgaff.com block page, a 212-byte check page on thalesgroup.com and imperva.com's homepage. text ```text giffgaff.com (block page): x-iinfo: 33-19324656-0 0NNN RT(1791880428778 239) q(0 -1 -1 17) r(0 -1) B12(11,19926025,0) U12 thalesgroup.com (check page): x-iinfo: 32-12293248-0 0NNN RT(1791880441127 82) q(0 -1 -1 0) r(0 -1) B10(4,289,0) U18 imperva.com (homepage): x-iinfo: 42-61436299-0 0CNN RT(1791880503843 83) q(0 -1 -1 1) r(0 -1) ``` We don't try to decode these values, but there's one part that stood out. All three block pages we got had `B12` in the header, the three check pages had `B10`, and the real pages had no `B` part. That's only nine responses from five websites, so we wouldn't rely on it. Still, it costs nothing to log it next to the status code. The incident ID only shows up on the block page. On both websites that blocked us, it started with the same number as the `incap_ses` cookie, like `975` in `975000500608034993` and `incap_ses_975_2118422` on giffgaff.com. The block page's HTML has both values too, in the `xinfo` and `incident_id` parameters of its iframe URL, next to a `cip` parameter with our IP address. So, make sure to log both values with every blocked request, along with the URL, the time and the proxy, as that's what we'd need to compare blocks later. We also saw the `incap_ses` cookie above, so let's take a look at Imperva's cookies next. ## How Do Imperva's Cookies and JavaScript Checks Work? Imperva sets its cookies on the very first response. All three websites from our table sent `visid_incap` and `incap_ses` cookies to curl, before we had any cookies of our own. Websites that run one of Imperva's JavaScript checks add another step. Their page loads a script from `/_Incapsula_Resource`, and once a browser runs it, a `reese84` or `___utmvc` cookie gets added. From then on, every request carries these cookies, and Imperva decides whether to serve the page, a CAPTCHA or a block page. mermaid ```mermaid flowchart TD A[Request to the site] --> B["Imperva sets visid_incap_*, incap_ses_* and nlbi_* cookies"] B --> C{"Does the site run a JavaScript check?"} C -->|no| F[Next request carries all the cookies] C -->|yes| D["Imperva's script runs in the browser (needs a real browser)"] D --> E["reese84 or ___utmvc cookie"] E --> F F --> G{Imperva decides} G --> H[Page content] G --> I[CAPTCHA page] G --> J[Block page with an incident ID] ``` Let's take a look at each of these cookies, starting with the ones that curl gets too. ### What Are the visid\_incap, incap\_ses and nlbi Cookies? The number in each cookie name changes between websites, like `2118422` on giffgaff.com and `2800108` on Coles. So, **our scraper should match these cookies by prefix**. Cookie notices on websites that use Imperva describe them like this: - `visid_incap_*` identifies the visitor across visits to the site. - `incap_ses_*` ties a visitor's requests to one session. - `nlbi_*` belongs to Imperva's load balancer. In the `Set-Cookie` headers we got, `visid_incap` expires on October 8, 2027, about a year later, and it's marked `HttpOnly`. As for `incap_ses`, it has no expiry date at all, so it only lasts for the session. The `nlbi` cookies are less consistent. Imperva's own website set `nlbi_2439_3105325` on one of our curl requests but not on the next one, while Chrome collected two `nlbi` cookies on Coles. Sending these cookies back doesn't get us through on its own, though. We requested giffgaff.com three times with curl, sending back the cookies from the first response, and got the block page every time. Still, **we send them back exactly as Imperva issued them, from the same client**, as a client that drops them starts every request as a new visitor. Our [cookie handling guide](https://scrapfly.io/blog/posts/how-to-handle-cookies-in-web-scraping) shows how to keep cookies across requests in Python. ### What Is the reese84 Cookie? `reese84` is the cookie set by Imperva's JavaScript check, which cookie notices describe as a cookie from Imperva Incapsula's security systems, used to evaluate web traffic and identify automated bot activity. On Coles, `reese84` was set on the `.www.coles.com.au` domain with an expiry of November 8, 2026, a month after our visit. Our curl requests to the same page only ever got `visid_incap` and `incap_ses`. The cookie didn't get our headless Chrome in, though. Three seconds after loading coles.com.au, it was already on Imperva's "Request unsuccessful" page, and nine seconds later it was still there. So, we keep running `is_imperva_block()` on every response, even with `reese84` in place. ### What Are \_\_\_utmvc and \_Incapsula\_Resource? `___utmvc` is the other cookie that Imperva's JavaScript checks can set, and like `reese84`, it only appears after the script has run in a browser. None of the three websites we tested set it. `_Incapsula_Resource` is a path on the protected website's own domain, where Imperva serves its scripts and block page iframes from. We saw it on the Coles check page, on the giffgaff.com block page and on imperva.com's normal homepage as well. With curl, we could download the 212-byte Coles page and the 109 KB script it loads, but no `reese84` cookie came back, and `curl_cffi` can't run that script either. Nodriver, SeleniumBase and Camoufox drive a real browser, so they can run it. Our headless Chrome got blocked with all of its Imperva cookies in place, so let's take a look at what else Imperva checks to identify web scrapers. ## How does Imperva identify web scrapers? To detect web scrapers, Imperva is taking advantage of many different analysis and fingerprinting techniques. Imperva is using combination of the above techniques to establish a unique fingerprint called a **trust score** for each connecting client. Based on the final trust score, Imperva decides whether to block the client, let it through or request an additional verification, such as [CAPTCHA](https://scrapfly.io/blog/posts/how-to-bypass-captcha-web-scraping). This process seems to be complex and daunting. However, if we study each component individually, we'll find that bypassing Imperva is possible. For more on these signals across eight anti-bot vendors, see our [anti-bot bypass guide](https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection). Let's take a look at each of these components, starting with the TLS handshake. ### TLS and JA3/JA4 Fingerprinting TLS (or SSL) fingerprinting is a modern technique of identifying a client based on the way the client and server negotiate an encrypted connection, called as the **JA3 fingerprint** (or its successor, JA4). For a secure connection (i.e., `https`), the encryption method needs to be negotiated between the client and server, as there are many different cipher and encryption options. So, if a connecting client has unusual capabilities, it can be easily identified as a bot. Libraries used in web scraping can have different encryption capabilities compared to a web browser. So, web scrapers can be easily identified by their TLS fingerprint even before the actual HTTP request is made. To avoid this, **use libraries and tools that are JA3 resistant**, such as `curl_cffi` or a real web browser. To validate this, see Scrapfly's [JA3 fingerprint tool](https://scrapfly.io/web-scraping-tools/ja3-fingerprint), which shows your connection fingerprint. For more, see our [JA3 and JA4 guide](https://scrapfly.io/blog/posts/ja3-ja4-tls-fingerprinting-guide-to-detection-and-evasion) and [introduction to TLS fingerprinting](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-tls#what-is-tls), which cover TLS fingerprinting in greater detail. ### IP Address Reputation The next step is the IP address analysis. Imperva has access to IP meta information databases that can be used to identify client's intentions and capabilities. For example, if the IP address belongs to a known proxy or datacenter service, it can be easily identified as a bot and get blocked. If the IP address is from a residential ISP, it is much more likely to be a human. Same for mobile networks. So, **use high-quality residential or mobile proxies** to avoid being detected. For a more in-depth look, see our [introduction to IP blocking](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-ip-addresses) and what IP metadata fields are used in bot detection. ### HTTP/2 and Header Details With the connection established the next step is HTTP connection analysis. Most of the natural web runs on HTTP/2 and HTTP/3 protocols (that's what web browsers prefer). This makes any HTTP/1.1 connection suspicious. Moreover, some HTTP libraries still use or default to HTTP/1.1, which is a dead giveaway that the request is automated. Python's [httpx](https://pypi.org/project/httpx/) supports HTTP/2, though it's not enabled by default. An HTTP/2 connection also has its own fingerprint, made of settings that differ between browsers and libraries. Scrapfly's [HTTP/2 fingerprint tool](https://scrapfly.io/web-scraping-tools/http2-fingerprint) shows what our client sends. Then, request header values and ordering can be used to identify the client. Web browser headers generation is well understood and reliable, so it's on web scrapers to match it. For example, web browsers send headers like `User-Agent`, `Sec-CH-UA`, `Sec-Fetch-Site`, `Origin` and `Referer`, and in a specific order to boot. So, **make sure to use HTTP/2 (or HTTP/3) and match header values and ordering of a real web browser**. For more, see our [HTTP/2 fingerprinting guide](https://scrapfly.io/blog/posts/http2-http3-fingerprinting-guide) and [request headers guide](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-headers#common-standard-headers). ### JavaScript Fingerprinting The final step is JavaScript fingerprinting. This is a very powerful technique that can be used to identify a client based on the way it executes JavaScript code. Since the server is allowed to execute almost any arbitrary JavaScript code on the client's machine, it can extract a lot of information about the client like: - JavaScript engine details - Hardware and operating system information - Web browser data and rendering capabilities - Canvas and WebGL fingerprinting - Audio context fingerprinting This combination of data can be used by Imperva to identify web scrapers. On Imperva websites, these checks run in the scripts that set the `reese84` and `___utmvc` cookies. To get around this, we have two approaches. The first one is to intercept the JavaScript fingerprinting and feed Imperva with fake data. However, this requires a lot of effort and is not very reliable, as any updates to the fingerprinting code will break our logic. Alternatively, we can use a **real web browser** to execute the JavaScript code. This is a much more reliable approach, as the browser runs whatever fingerprinting code Imperva sends. So, **browser automation is an easier and more reliable way to handle JavaScript fingerprinting**, and it's the approach that Nodriver, SeleniumBase and Camoufox take. ### Behavioral Analysis Even if we address all of these detection methods, Imperva can still identify scrapers due to continuous behavioral analysis. As Imperva is tracking all connection details and patterns, it can use this information to adjust the trust score constantly, which can lead to blocking or CAPTCHA challenges. So, it's important to **distribute the scraping load through multiple agents** using proxies and different fingerprint configurations. For example, when scraping using browser automation tools, it's important to use a collection of different profiles like screen size, operating system, rendering capabilities together with IP proxies. Now that we've covered how Imperva detects web scrapers, let's take a look at the tools we can use to bypass it. Scrapfly #### Need to bypass anti-bot protection? Scrapfly's Unblocker handles Cloudflare, DataDome, and more — automatically. [Try Free →](https://scrapfly.io/register)## How to Bypass Imperva? We can see that there's a lot going on when it comes to Imperva's anti-bot technology and since it's using score based approach, we don't necessarily need to bypass all of the detection methods perfectly. To quickly summarize, here's where scrapers can be improved to avoid detection: - Use high quality residential or mobile proxies - Use HTTP2 (or later) version for all requests - Match request header values and ordering of a real web browser - Use real browser automation to generate JavaScript fingerprints - Distribute web scraper traffic through multiple agents While Imperva bypass at scale requires a lot of technical reverse engineering knowledge, we can use existing tools to have a fair amount of success bypassing it. Note that as Imperva is developing and improving their methods it's important to stay in touch with web scraping tool and library updates. To bypass HTTP, TLS and JavaScript fingerprinting, we can use a real web browser like Chrome or Firefox through Selenium, Playwright or Puppeteer automation libraries. However, these browsers are easily detected by anti-bot services like Imperva, as these libraries leak their usage through default fingerprints like `navigator.webdriver`. This is where community patches like [undetected-chromedriver](https://scrapfly.io/blog/posts/web-scraping-without-blocking-using-undetected-chromedriver) for Selenium, [puppeteer-extra-plugin-stealth](https://scrapfly.io/blog/posts/puppeteer-stealth-complete-guide) for Puppeteer and [playwright-stealth](https://scrapfly.io/blog/posts/playwright-stealth-bypass-bot-detection) for Playwright can be helpful. They patch the browser features to make it more difficult to differentiate from a real web browser. Now that we're familiar with all methods Imperva is using to detect web scrapers, let's take a look at the tools we can use to bypass it. ### Which Imperva Bypass Tool Fixes Which Layer? Let's take a look at existing options and how they play a role in Imperva bypass. Here are the tools we'll use in this guide, with the versions we ran. | Tool | Version we ran | Runs page JavaScript | Browser engine | Python | |---|---|---|---|---| | `curl_cffi` | 0.16.3 | No | None | 3.10+ | | Nodriver | 0.50.3 | Yes | Chrome | 3.13 or older (0.50.6 runs on 3.14) | | SeleniumBase (UC Mode, CDP Mode) | 4.54.13 | Yes | Chrome | 3.10+ | | Camoufox | 0.5.6 | Yes | Firefox | 3.10+ | | Scrapfly Web Scraping API (Unblocker) | Hosted service | When needed | Managed | SDKs for 4 languages | I sent `requests`, `curl_cffi` and headless Chrome 154 to the API behind Scrapfly's [JA3 fingerprint tool](https://scrapfly.io/web-scraping-tools/ja3-fingerprint), and here are the JA4 fingerprints it returned. text ```text requests: t13d1712ht_ab0a1bf427ad_882d495ac381 curl_cffi: t13d1516h2_8daaf6152771_806a8c22fdea chrome: t13d1517h2_8daaf6152771_cb7bf5808d99 ``` We can see that `curl_cffi` has the same cipher part as Chrome, `8daaf6152771`, while `requests` offers different ciphers and no HTTP/2 at all. So, `curl_cffi` covers the TLS and HTTP/2 layer. However, it can't run JavaScript, and that's where browsers come in. As Imperva is using JavaScript fingerprinting and challenges to detect web scrapers, using headless browsers is often a requirement. Headless browsers can be used to execute JavaScript challenges and Imperva fingerprinting, which can be used to bypass the anti-bot systems. Alternatively, each JavaScript challenge and fingerprint has to be reverse engineered and solved manually, which is a very tough task even for seasoned developers. However, these browsers are easily detected by Imperva when they're being run in headless mode. Here's how the four clients did against our three Imperva websites on October 9, with Nodriver and Camoufox both running headless. | Site | curl | `curl_cffi` 0.16.3 | Nodriver 0.50.3, headless | Camoufox 0.5.6, headless | |---|---|---|---|---| | giffgaff.com | Block page | Block page | Block page | Block page | | coles.com.au | 212-byte check page | 212-byte check page | Block page, with `reese84` set | `page.goto()` timed out after 30 seconds | | imperva.com | Real homepage | Real homepage | Real homepage | Real homepage, with `reese84` set | None of the four clients got past giffgaff.com, and only imperva.com gave everyone its real homepage. On Coles, Nodriver ran the check script and still got a block page, while Camoufox timed out after 30 seconds. So, **on a real target, we run the browsers headful and behind residential proxies**. None of these runs used a proxy either, so let's take a look at residential proxies first. ### Start With Residential Proxies and One Session per IP As Imperva is using IP address analysis to determine the trust score, using high-quality residential or mobile proxies can help bypass the IP address fingerprinting. Residential proxies are real IP addresses assigned by internet providers to retail individuals, making them look like real users. Web scraping APIs like Scrapfly already use high quality proxies by default as that's often the best way to bypass anti-scraping protection at scale. New to proxies? See our [introduction to proxies in web scraping](https://scrapfly.io/blog/posts/introduction-to-proxies-in-web-scraping) and how to evaluate them. Moreover, web scraping at scale requires rotating proxies. This is to prevent IP address blocking when the requests' rate exceeds the defined limits by splitting the load across multiple IP addresses. However, rotating residential proxies can have **session persistency issues** with maintaining the same IP address for long periods. For example, if we are scraping web data at the end of a long process, the proxy manager can change the IP address before we reach the end. With Imperva, the `incap_ses` cookie we saw earlier had no expiry date, so it only lasts for the session. Hence, **we keep the same IP address and cookie jar for the whole session**. A blocked IP address can also recover after a while. Thalesgroup.com blocked our `curl_cffi` requests on the evening of October 12, and the next morning, the same IP address got the real 181 KB homepage again. If we use Camoufox, its [geoip option](https://camoufox.com/python/geoip/) sets the browser's location, timezone and locale from the proxy's IP address, so a German IP doesn't come with a US timezone. Now that we have the proxies, let's start with the first tool, `curl_cffi`. ### Use curl\_cffi for Imperva's TLS and HTTP/2 Checks [curl\_cffi](https://github.com/lexiforest/curl_cffi) is a Python HTTP client library built on top of Curl Impersonate. Since Curl Impersonate modifies the request configuration to mimic normal browser ones, we can use it to scrape without getting blocked! First, let's install `curl_cffi` using the `pip` command. We used version 0.16.3 from September 2, 2026, where `impersonate="chrome"` points to the `chrome150` target. bash ```bash pip install curl_cffi ``` We'll also save a small `imperva_check.py` file with two helpers, as the browser examples and the troubleshooting section below import them. python ```python from curl_cffi import requests IMPERVA_COOKIES = ("visid_incap_", "incap_ses_", "nlbi_", "reese84", "___utmvc") IMPERVA_MARKERS = ("incapsula incident id", "_incapsula_resource", "powered by incapsula") def imperva_cookies(cookie_names): return sorted({name for name in cookie_names if name.startswith(IMPERVA_COOKIES)}) def is_imperva_block(html, expected_text): has_marker = any(marker in html.lower() for marker in IMPERVA_MARKERS) return has_marker and expected_text not in html def fetch(url, expected_text): with requests.Session(impersonate="chrome") as session: response = session.get(url, timeout=30) print("status:", response.status_code) print("x-iinfo header:", "x-iinfo" in response.headers) print("imperva cookies:", imperva_cookies(session.cookies.keys())) print("blocked:", is_imperva_block(response.text, expected_text)) return response if __name__ == "__main__": fetch("https://web-scraping.dev/products", expected_text="Box of Chocolate Candy") ``` Here, `imperva_cookies()` picks Imperva's cookies out of the cookie jar, and `is_imperva_block()` flags a page that has an Imperva marker but not the text we expect. To evaluate the `curl_cffi` capabilities, let's try to scrape giffgaff.com with it. All we have to do is call `fetch()` with the target website URL and the text we expect on the page. python ```python from imperva_check import fetch fetch("https://www.giffgaff.com/", expected_text="giffgaff") ``` Here is what we got. text ```text status: 200 x-iinfo header: True imperva cookies: ['incap_ses_975_2118422', 'visid_incap_2118422'] blocked: True ``` The website detected our request as automated and we got blocked, even with a 200 status code and Chrome's TLS fingerprint. Let's try imperva.com next, with "Imperva" as the expected text. python ```python from imperva_check import fetch fetch("https://www.imperva.com/", expected_text="Imperva") ``` text ```text status: 200 x-iinfo header: True imperva cookies: ['incap_ses_975_2439', 'nlbi_2439_3105325', 'visid_incap_2439'] blocked: False ``` The website recognized our request as coming from a regular browser and we were allowed! `curl_cffi` can be efficient at mimicking normal browser requests. However, its bypass rate can be limited. This is due to the challenges that websites use to verify human traffic from bots. For example, Coles sent `curl_cffi` a 212-byte page with one script from `/_Incapsula_Resource`. And since JavaScript isn't enabled within our HTTP request, we couldn't get past it. For more on curl itself, see our [curl-impersonate guide](https://scrapfly.io/blog/posts/curl-impersonate-scrape-chrome-firefox-tls-http2-fingerprint). Let's have a look at a real browser that can run that script, Nodriver! ### Try Nodriver [Nodriver](https://github.com/ultrafunkamsterdam/nodriver) is the official successor of the Undetected ChromeDriver, a modified Web Driver for Selenium. Unlike its predecessor, it talks to Chrome over the DevTools Protocol, without Selenium or a web driver. Before proceeding to web scraping with Nodriver, let's install it. It can be installed using the pip terminal command, and it uses the Chrome browser that's already on the machine. bash ```bash pip install nodriver ``` Note that Nodriver 0.50.3, which we used here, fails to import on Python 3.14 with a `SyntaxError` in `nodriver/cdp/network.py`. It works on Python 3.13, and release 0.50.6 from October 8, 2026 fixes it. Both Nodriver and its fork [Zendriver](https://pypi.org/project/zendriver/) are licensed AGPL-3.0, which matters if the scraper ships inside a product. To confirm its capabilities, let's try Coles, the website that sent `curl_cffi` its 212-byte check page. python ```python import nodriver as uc from imperva_check import imperva_cookies, is_imperva_block async def main(): browser = await uc.start(headless=True) page = await browser.get("https://www.coles.com.au/") await page.sleep(5) html = await page.get_content() cookies = await browser.cookies.get_all() print("imperva cookies:", imperva_cookies(cookie.name for cookie in cookies)) print("blocked:", is_imperva_block(html, "Coles")) browser.stop() if __name__ == "__main__": uc.loop().run_until_complete(main()) ``` Here, we start a headless Chrome, go to the target website and wait 5 seconds for Imperva's script to finish. Here is what we got. text ```text imperva cookies: ['___utmvc', 'incap_ses_340_2800108', 'visid_incap_2800108'] blocked: True ``` We can see that the website detected us, even though Chrome ran the check script and got the `___utmvc` cookie. On October 9, the same script got a `reese84` cookie on Coles instead, and it was blocked then too. In headless mode, Nodriver sends `HeadlessChrome/154.0.0.0` as its User-Agent. So, **on a real target, we run it with `headless=False` and residential proxies**. For more on its predecessor, see our [undetected-chromedriver guide](https://scrapfly.io/blog/posts/web-scraping-without-blocking-using-undetected-chromedriver). Now, let's try the same page with SeleniumBase. ### Try SeleniumBase: Selenium With UC Mode and CDP Mode [SeleniumBase](https://github.com/seleniumbase/SeleniumBase) is a framework built on Selenium, and its [UC Mode](https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/uc_mode.md) is based on the Undetected ChromeDriver. From there, `activate_cdp_mode()` switches the session to [CDP Mode](https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/ReadMe.md), where Chrome is driven over the DevTools Protocol. It can be installed using the pip terminal command. We used 4.54.13, which downloaded chromedriver 154 on its first run to match our Chrome. bash ```bash pip install seleniumbase ``` python ```python from seleniumbase import SB from imperva_check import imperva_cookies, is_imperva_block with SB(uc=True, headless=True) as sb: sb.activate_cdp_mode("https://www.coles.com.au/") sb.sleep(5) html = sb.cdp.get_page_source() cookies = sb.cdp.get_all_cookies() print("imperva cookies:", imperva_cookies(cookie.name for cookie in cookies)) print("blocked:", is_imperva_block(html, "Coles")) print("next.js app:", "__NEXT_DATA__" in html) ``` Here, `SB(uc=True, headless=True)` starts Chrome in UC Mode, and `activate_cdp_mode()` opens Coles in CDP Mode. Then, we read the page source and cookies through `sb.cdp`. Here is what we got. text ```text imperva cookies: ['incap_ses_340_2800108', 'visid_incap_2800108'] blocked: False next.js app: True ``` We successfully avoided Imperva's block page using SeleniumBase, from the same IP address that Nodriver got blocked on less than a minute earlier. The page was Coles' own Next.js app, with `__NEXT_DATA__` in the HTML. Its headless Chrome sent a normal `Chrome/154.0.0.0` User-Agent, without the "Headless" part. In an earlier run, though, the page still had no visible text after 5 seconds, so we'd wait longer before parsing it. For a deeper look, see our [SeleniumBase guide](https://scrapfly.io/blog/posts/guide-to-seleniumbase-better-selenium). Both of these run Chrome, so let's try a Firefox-based browser next, Camoufox. ### Try Camoufox [Camoufox](https://github.com/daijro/camoufox) is a Firefox-based browser for web scraping. Instead of patching an automated browser from the outside, it injects device characteristics like the OS, CPU, fonts, screen size and WebGL into Firefox itself. Its Python package wraps Playwright, and it takes two commands to install, as the second one downloads the browser. That download is about 1.3 GB on Windows. bash ```bash pip install -U "camoufox[geoip]" python -m camoufox fetch ``` We used Camoufox 0.5.6, which generates its fingerprints with BrowserForge, while the [PyPI description](https://pypi.org/project/camoufox/) of 0.5.7 and later names fpgen. By October 6, 2026, the newest package was 0.5.8. Let's try imperva.com with it, in headless mode. python ```python from camoufox.sync_api import Camoufox from imperva_check import imperva_cookies, is_imperva_block with Camoufox(os="windows", humanize=True, headless=True) as browser: page = browser.new_page() page.goto("https://www.imperva.com/") page.wait_for_timeout(5000) html = page.content() cookies = page.context.cookies() print("imperva cookies:", imperva_cookies(cookie["name"] for cookie in cookies)) print("blocked:", is_imperva_block(html, "Imperva")) ``` In this example, `os` sets the operating system that the fingerprint claims, and `humanize` makes the cursor movement look human, according to the [usage docs](https://camoufox.com/python/usage/). Here is what we got on October 9. text ```text imperva cookies: ['incap_ses_975_2439', 'nlbi_2439', 'nlbi_2439_2147483392', 'nlbi_2439_3105325', 'reese84', 'visid_incap_2439'] blocked: False ``` We can see that Camoufox ran Imperva's script and got the `reese84` cookie along with the real homepage, which no other client did on imperva.com. However, it still got the block page on giffgaff.com, and on coles.com.au, `page.goto()` timed out after 30 seconds. To run it in the headful mode, we can simply set `headless=False`. With a proxy, we also pass `geoip=True`, so the timezone and locale match the proxy's IP address. Our [stealth browser comparison](https://scrapfly.io/blog/posts/best-stealth-browsers) covers Camoufox and the Chrome options in more detail. Running a full browser for every page is slow, though, so let's see if `curl_cffi` can take over. ### Can You Reuse Imperva Cookies in curl\_cffi? Another advantage of using `curl_cffi` is reusing sessions. We can get the cookies of a successful browser session and then reuse them with future requests, which is what Scrapfly's [session feature](https://scrapfly.io/docs/scrape-api/session) does for us. Let's try it on Coles, where SeleniumBase got through and `curl_cffi` on its own got the 212-byte check page. python ```python from curl_cffi import requests from seleniumbase import SB from imperva_check import imperva_cookies, is_imperva_block with SB(uc=True, headless=True) as sb: sb.activate_cdp_mode("https://www.coles.com.au/") sb.sleep(5) cookies = {cookie.name: cookie.value for cookie in sb.cdp.get_all_cookies()} print("browser cookies:", imperva_cookies(cookies)) with requests.Session(impersonate="chrome") as session: session.cookies.update(cookies) response = session.get("https://www.coles.com.au/", timeout=30) print("curl_cffi:", response.status_code, len(response.content), "bytes") print("blocked:", is_imperva_block(response.text, "Coles")) ``` Here, SeleniumBase loads the page and passes its cookies to a `curl_cffi` session, which requests the same page. Here is what we got. text ```text browser cookies: ['incap_ses_62_2800108', 'nlbi_2800108_2147483392', 'nlbi_2800108_2670698', 'nlbi_2800108_3037207', 'reese84', 'visid_incap_2800108'] curl_cffi: 200 354444 bytes blocked: False ``` With the browser's cookies, `curl_cffi` got the real Coles homepage, titled "Shop groceries online \| Coles Supermarket", instead of the 212-byte check page. Note that this only works with cookies from a session that got the real page. When I sent giffgaff.com its own cookies back with curl, three requests in a row, every one got the block page. So, **we move all of Imperva's cookies together**, keep the same IP address, and go back to the browser as soon as `is_imperva_block()` returns `True`. Imperva keeps scoring every request after the first one, so let's take a look at browsing like a regular visitor. ### Browse Like a Visitor: Warm-Up, Pacing and Mouse Movement To bypass behavior analysis adjusting scraper behavior to appear more natural can drastically increase Imperva trust scores. In reality, most human users don't visit product URLs directly. They often explore websites in steps like: - Start with homepage - Browse product categories - Search for product - View product page Prefixing scraping logic with this warmup behavior can make the scraper appear more human-like and avoid behavior analysis detection. For sustained web scraping and Imperva bypass, these browsers should always be remixed with different, realistic fingerprint profiles, as screen resolution, operating system and browser type all play an important role in Imperva's trust score. Each headless browser library can be configured to use different resolution and rendering capabilities. Distributing scraping through multiple realistic browser configurations can prevent Imperva from detecting the scraper. Mimicking mouse moves and keyboard clicks helps too, and Camoufox's `humanize=True` takes care of the mouse part for us. Starting a fresh session helps as well. On thalesgroup.com, our paced `curl_cffi` session got the Pardon challenge on 5 of its 10 pages, and a new one from the same IP address got through right after. For more see ScrapFly's [browser fingerprint tool](https://scrapfly.io/web-scraping-tools/browser-fingerprint) to see how your browser looks like to Imperva. This tool can be used to collect different browser fingerprints from real web browsers which can be used for scraping. Some requests will still get blocked after all of this, so let's go through the common Imperva blocks next. ## Troubleshooting Common Imperva Blocks Most of Imperva bot blocks result in HTTP status codes 400-500, 403 being the most common one. However, when I requested five Imperva websites with Python's `requests` and `curl_cffi` on October 12, all of them answered with a 200. | Website | `requests` | `curl_cffi` | |---|---|---| | giffgaff.com | 958-byte block page | 955-byte block page | | coles.com.au | 212-byte check page | 212-byte check page | | pokemoncenter.com | 1,048-byte block page | Real homepage, 637 KB | | thalesgroup.com | 212-byte check page | Real homepage, 181 KB | | imperva.com | Real homepage, 249 KB | Real homepage, 249 KB | From the above, we can see that the status code doesn't tell us much. The block and check pages were between 212 bytes and about 1 KB, while the real homepages were 181 KB or more. Finding out why a request got blocked takes more work. For that, I picked pokemoncenter.com, since `requests` got blocked there while `curl_cffi` got through. ### Blocked on the First Request For example, let's attempt to scrape the pokemoncenter.com homepage using the regular Python `requests`. We will check the response with the `is_imperva_block()` function from our `curl_cffi` example, with "Pokémon Center" as the expected text. python ```python import requests from imperva_check import is_imperva_block response = requests.get("https://www.pokemoncenter.com/") print(response.status_code, len(response.content), "bytes") print("blocked:", is_imperva_block(response.text, "Pokémon Center")) ``` Here is what we got. text ```text 200 1048 bytes blocked: True ``` The website detected our request as automated and we got blocked, even with a 200 status code. Our request still has the default `python-requests` User-Agent, so let's replace it with a Chrome one. python ```python import requests from imperva_check import is_imperva_block headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Safari/537.36", } response = requests.get("https://www.pokemoncenter.com/", headers=headers) print(response.status_code, len(response.content), "bytes") print("blocked:", is_imperva_block(response.text, "Pokémon Center")) ``` text ```text 200 6183 bytes blocked: True ``` This time, we got a different page. It is titled "Pardon Our Interruption", and it came with Imperva's `X-Iinfo` header and `incap` cookies. Here is its text, trimmed a bit. text ```text Pardon Our Interruption As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen: You've disabled JavaScript in your web browser. You're a power user moving through this website with super-human speed. You've disabled cookies in your web browser. A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. ``` This page is basically an Imperva JavaScript challenge. In a real browser, it runs a challenge script and reloads itself, and the text above only shows up if the script doesn't finish within 10 seconds. Our scraper can't run JavaScript, so it can't pass this challenge. However, a real browser also sends a lot more headers than just the User-Agent, so let's add the rest of the headers Chrome sends. python ```python import requests from imperva_check import is_imperva_block headers = { "sec-ch-ua": '"Chromium";v="150", "Google Chrome";v="150", "Not_A Brand";v="99"', "sec-ch-ua-mobile": "?0", "sec-ch-ua-platform": '"Windows"', "upgrade-insecure-requests": "1", "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Safari/537.36", "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "sec-fetch-site": "none", "sec-fetch-mode": "navigate", "sec-fetch-user": "?1", "sec-fetch-dest": "document", "accept-encoding": "gzip, deflate", "accept-language": "en-US,en;q=0.9", } response = requests.get("https://www.pokemoncenter.com/", headers=headers) print(response.status_code, len(response.content), "bytes") print("blocked:", is_imperva_block(response.text, "Pokémon Center")) ``` text ```text 200 637016 bytes blocked: False ``` We got the full homepage, and that's with Python's own TLS fingerprint! So, on this website, **a complete set of browser headers was enough to get past the first request**. Note that we left `br` and `zstd` out of `Accept-Encoding`, even though Chrome sends them, since `requests` needs the extra `brotli` and `zstandard` packages to decode them. For more on these headers and what each of them means, see our [request headers guide](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-headers#common-standard-headers). Imperva can also block with a 403 status code. For example, `curl_cffi` with the old `chrome99` target got a 403 with an 858-byte body three times out of five that evening. Every response we got had an `X-Iinfo` header, including the 403. For a 403 without one, see our [403 guide](https://scrapfly.io/blog/posts/403-forbidden-web-scraping), as Imperva most likely isn't the one blocking the request. Earlier that evening, `curl_cffi` with the `chrome` target got the real homepage from this website too, before it started getting the Pardon page. So, let's take a look at blocks that show up after a few requests. ### Blocked After a Few Pages Imperva keeps scoring the connection after the first request. This means that the trust score can be decreased over time, and a scraper that got through the first pages can still get blocked later on. This is what happened with `curl_cffi` on pokemoncenter.com. It got the real homepage at first, and after about 20 requests from the same IP address, it started getting the Pardon page. Imperva can also detect the requests as coming from bots if the requesting rate is high through a narrow time window. Therefore, hiding your IP address by splitting the requests across different IPs can help avoid Imperva blocking while scraping. If the website answers with a 429 status code, **we slow down and wait before retrying**, at least as long as its `Retry-After` header says. For more, see our [guide to rate limiting async requests](https://scrapfly.io/blog/posts/how-to-rate-limit-asynchronous-python-requests). Once we have our proxies rolling eventually they will start being blocked. Some will be more successful than others and some blocked ones will at some point recover. Hence, **we rest a blocked proxy for a while**. In some cases, Imperva doesn't block these requests and shows a challenge page instead, so let's take a look at those next. ### CAPTCHA Pages That Keep Coming Back Some of Imperva's anti-bot protection measures require solving CAPTCHA challenges, like the security check page with a reCAPTCHA checkbox we saw earlier. However, the best way to bypass Imperva CAPTCHA is to prevent it from occurring in the first place! Theoretically, it's possible to reverse engineer the computational JavaScript challenges and solve them using scripts. However, any change to the challenge algorithms makes the solving script outdated. While cookies in web scraping don't play a critical role in HTTP fingerprinting, they ensure the same website behavior of browser navigation when scraping. So, **we keep Imperva's cookies between requests**, or the same check comes back on every page. There are a few key differences between headless browsers and regular ones. Antibot solutions, like Imperva, rely on these differences to detect headless browser usage. For example, when I loaded pokemoncenter.com three times with Nodriver in headless mode, every visit ended on Imperva's block page. Chrome did run the challenge script, as it got the `reese84` cookie on the first visit. Moreover, the default HTTP details of HTTP clients and browser automation tools can leak their usage, such as the default User-Agent of each client. In our case, headless Chrome was sending `HeadlessChrome/154.0.0.0` as its User-Agent. Once I replaced it with a normal Chrome User-Agent, the Imperva block page was gone. Instead, we got a 1,866-byte page with a "DataDome Device Check" iframe, as pokemoncenter.com runs DataDome as well. So, **we check whose challenge it is before changing anything else**. Our `is_imperva_block()` didn't catch the DataDome page, as it has no Imperva markers, and DataDome's checks are covered in our [DataDome bypass guide](https://scrapfly.io/blog/posts/how-to-bypass-datadome-anti-scraping). Next, let's take a look at a scraper that worked fine and then started getting blocked. ### When a Working Scraper Starts Getting Blocked Sometimes, a scraper that worked fine starts getting blocked without any change in its code. This is exactly what happened with `curl_cffi` on pokemoncenter.com, which got the real homepage at first and then the Pardon page after about 20 requests. Let's run the `fetch()` function from our `curl_cffi` example on it again, about half an hour after the last request. python ```python from imperva_check import fetch fetch("https://www.pokemoncenter.com/", expected_text="Pokémon Center") ``` Here is what we got. text ```text status: 200 x-iinfo header: True imperva cookies: ['incap_ses_334_2682446', 'visid_incap_2682446'] blocked: True ``` The website is still blocking our requests. The first thing to check is whether our tools are up to date, so let's see which `curl_cffi` version we have and which Chrome version it impersonates. python ```python import curl_cffi from curl_cffi.requests.impersonate import DEFAULT_CHROME print(curl_cffi.__version__, DEFAULT_CHROME) ``` text ```text 0.16.3 chrome150 ``` At the time of writing, 0.16.3 is also the newest `curl_cffi` release on PyPI, so updating it isn't an option here. The browser tools did get new releases, though. Here's what PyPI listed on October 12: - Nodriver 0.50.6, released on October 8, 2026 - SeleniumBase 4.56.1, released on October 12, 2026 - Camoufox 0.5.8, released on October 6, 2026 All three are newer than the versions we tested earlier. So, when a browser scraper starts getting blocked, **we update the browser tool before changing anything else**. Our `curl_cffi` is already up to date, so what's left is the IP address. Let's run the same `fetch()` on imperva.com, from the same IP address and about a minute later. python ```python from imperva_check import fetch fetch("https://www.imperva.com/", expected_text="Imperva") ``` text ```text status: 200 x-iinfo header: True imperva cookies: ['incap_ses_975_2439', 'visid_incap_2439'] blocked: False ``` imperva.com still lets us in, so our IP address isn't blocked everywhere. Each website runs its own Imperva setup, as the different numbers in the cookie names show, `2682446` on pokemoncenter.com and `2439` on imperva.com. Our requests didn't send any cookies either, as each `fetch()` call starts a new session. So, whatever pokemoncenter.com remembers about us is tied to our IP address and our client. I ran these requests without a proxy, so I couldn't switch the IP address. With a proxy pool, **we'd retest with a new IP address and a fresh session** before changing anything in the code. Let's have a look at a better alternative for scraping Imperva websites without getting blocked! ## Bypass Imperva with ScrapFly Bypassing Imperva anti-bot while possible is very difficult, so let Scrapfly's [Imperva bypass API](https://scrapfly.io/bypass/incapsula) do it for you! ScrapFly provides web scraping, screenshot, and extraction APIs for data collection at scale. Its [Web Scraping API](https://scrapfly.io/products/web-scraping-api) is a single HTTP endpoint with a **99.99% success rate**, running on **130M+ proxies in 120+ countries**. Each product is equipped with an automatic bypass for any anti-bot system and we achieve this by: - Bypassing Cloudflare, DataDome, PerimeterX, Akamai and 90+ other bot systems with the [Unblocker](https://scrapfly.io/docs/scrape-api/unblocker). - Rotating residential and datacenter [proxies](https://scrapfly.io/docs/scrape-api/proxy), with geo-targeting by country and ASN. - Maintaining a fleet of real cloud browsers for [JavaScript rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) of SPAs and dynamic pages. - Keeping the cookies, headers and IP address of each [session](https://scrapfly.io/docs/scrape-api/session) consistent across multi-step flows. On top of that, the Web Scraping API also comes with: - [Browser automation scenarios](https://scrapfly.io/docs/scrape-api/javascript-scenario) to scroll, click, fill forms and wait for elements, without managing a browser fleet. - Pages returned as HTML, JSON, clean text or LLM-ready Markdown through [format conversion](https://scrapfly.io/docs/scrape-api/getting-started#api_param_format). - [Smart caching](https://scrapfly.io/docs/scrape-api/getting-started#api_param_cache), which saves successful responses and cuts the cost of repeat scraping jobs. - [Python](https://scrapfly.io/docs/sdk/python), [TypeScript](https://scrapfly.io/docs/sdk/typescript) and [Scrapy](https://scrapfly.io/docs/sdk/scrapy) SDKs, plus [no-code integrations](https://scrapfly.io/docs/integration/getting-started) like Make, n8n, Zapier, LangChain and LlamaIndex. We've been doing this publicly since 2020 with the best bypass on the market! It takes Scrapfly several full-time engineers to maintain this system, so you don't have to! ### Power your scraping with Scrapfly Forget about getting blocked. Scrapfly handles anti-bot bypasses, browser rendering, and proxy rotation so you can focus on the data. [Try for FREE!](https://scrapfly.io/register) For example, to scrape pages protected by Imperva using the [Web Scraping API](https://scrapfly.io/products/web-scraping-api), all we need to do is enable the Unblocker feature. The Unblocker was previously called Anti Scraping Protection (ASP). The `asp` parameter still works and isn't deprecated, while `unblocker=True` is the new name for the same switch. According to the [Unblocker docs](https://scrapfly.io/docs/scrape-api/unblocker), it can switch to a better proxy pool, turn on a browser when a page needs JavaScript and adjust the headers. It also "auto handles session usage and reuses challenge cookies". Note that these adjustments can raise the credit cost of a request. Scrapfly has SDKs for Python, TypeScript, Go and Rust, and the [Python SDK](https://scrapfly.io/docs/sdk/python) can be installed using `pip`. bash ```bash pip install scrapfly-sdk ``` Then, to fetch a page with the Unblocker enabled, all we have to do is set the `asp` parameter. python ```python from scrapfly import ScrapflyClient, ScrapeConfig scrapfly = ScrapflyClient(key="YOUR API KEY") result = scrapfly.scrape(ScrapeConfig( url="https://web-scraping.dev/products", # Unblocker (formerly ASP): detects the anti-bot system and handles its challenges asp=True, # we can also enable headless browsers to render web apps and javascript powered pages render_js=True, # and set proxies by country like France country="FR", # and proxy type like residential: proxy_pool=ScrapeConfig.PUBLIC_RESIDENTIAL_POOL, )) print(result.scrape_result) ``` Here, `asp=True` turns the Unblocker on, `render_js` enables a headless browser, and `country` with `proxy_pool` picks residential proxies in France. ## FAQ Is it legal to scrape Imperva-protected pages?It depends on the country, the data we collect and how we access the pages. Generally, Imperva protection doesn't make public data private, but getting past a bot check doesn't make scraping legal either. Is it possible to bypass Imperva?Yes, individual sessions can get through with a real browser, a clean residential IP and one consistent session. However, there's no permanent setup, as Imperva scores every request and each website configures it differently. So, we retest the scraper whenever blocks start to climb, and keep running `is_imperva_block()` on every response so that a block never passes as data. How do I identify if a website is using Imperva protection?Look for the `visid_incap_*` and `incap_ses_*` [cookies](https://scrapfly.io/blog/posts/how-to-handle-cookies-in-web-scraping), the `X-Iinfo` header and `/_Incapsula_Resource` paths, which confirm that Imperva is there. An "Incapsula incident ID" page means that the request was blocked. Note that the first three show up on normal pages too. For more details, see our section on [recognizing Imperva](#how-do-you-know-a-site-uses-imperva) above. Can I bypass Imperva using only HTTP requests without browsers?Yes, but only on pages where Imperva doesn't serve a JavaScript check. On those pages, [curl\_cffi](#use-curl-cffi-for-imperva-s-tls-and-http-2-checks) can fix the TLS and HTTP/2 fingerprint, which is enough. Where Imperva uses `reese84` or `___utmvc`, an HTTP client can't run the script that sets the cookie. So, that step needs a real browser, even if `curl_cffi` takes over afterwards. What is the reese84 cookie?The `reese84` cookie is set by Imperva's JavaScript check. Websites' cookie notices describe it as an Imperva Incapsula cookie that evaluates web traffic to identify automated bots. It only gets set after Imperva's script runs in a browser. However, having it doesn't guarantee that the next response is the actual page, so the response body still needs checking. What are some other anti-bot services?There are many other anti-bot WAF services like [Cloudflare](https://scrapfly.io/blog/posts/how-to-bypass-cloudflare-anti-scraping), [Akamai](https://scrapfly.io/blog/posts/how-to-bypass-akamai-anti-scraping), [DataDome](https://scrapfly.io/blog/posts/how-to-bypass-datadome-anti-scraping), [PerimeterX](https://scrapfly.io/blog/posts/how-to-bypass-perimeterx-human-anti-scraping) and [Kasada](https://scrapfly.io/blog/posts/how-to-bypass-kasada-anti-scraping-waf). They function very similarly to Imperva's Incapsula, though each one uses its own cookies and challenges. To find out which one a website uses, the [Scrapfly Antibot Detector](https://scrapfly.io/products/antibot-detector) checks it for us. ## Summary In this guide, we've taken a look at how to bypass Incapsula (now known as Imperva) when web scraping public pages. Imperva's cookies and block pages still use the Incapsula name. The `visid_incap` and `incap_ses` cookies and the `X-Iinfo` header only show that Imperva is in front of a website, and we got all three from imperva.com along with its real homepage. Our curl request to giffgaff.com also came back with a 200 status code, but the page was Imperva's "Request unsuccessful" block page. So, **we check the body for the incident ID and for the content we expected**. We've also taken a look at the detection methods Imperva is using and how can we address each one of them in our scraper code. We saw that using residential proxies with one session per IP and patching common fingerprinting techniques can vastly improve trust scores when it comes to Imperva's bot blocking. `curl_cffi` patches the TLS and HTTP/2 fingerprint, though it can't run JavaScript. Where a website uses `reese84` or `___utmvc`, Imperva's script has to run in a real browser like Nodriver, SeleniumBase or Camoufox. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents - [Key Takeaways](#key-takeaways) - [What is Imperva (aka Incapsula)?](#what-is-imperva-aka-incapsula) - [How Do You Know a Site Uses Imperva?](#how-do-you-know-a-site-uses-imperva) - [Why Can an Imperva Block Return HTTP 200?](#why-can-an-imperva-block-return-http-200) - [What Do X-Iinfo and the Incapsula Incident ID Tell You?](#what-do-x-iinfo-and-the-incapsula-incident-id-tell-you) - [How Do Imperva's Cookies and JavaScript Checks Work?](#how-do-imperva-s-cookies-and-javascript-checks-work) - [What Are the visid\_incap, incap\_ses and nlbi Cookies?](#what-are-the-visid-incap-incap-ses-and-nlbi-cookies) - [What Is the reese84 Cookie?](#what-is-the-reese84-cookie) - [What Are \_\_\_utmvc and \_Incapsula\_Resource?](#what-are-utmvc-and-incapsula-resource) - [How does Imperva identify web scrapers?](#how-does-imperva-identify-web-scrapers) - [TLS and JA3/JA4 Fingerprinting](#tls-and-ja3-ja4-fingerprinting) - [IP Address Reputation](#ip-address-reputation) - [HTTP/2 and Header Details](#http-2-and-header-details) - [JavaScript Fingerprinting](#javascript-fingerprinting) - [Behavioral Analysis](#behavioral-analysis) - [How to Bypass Imperva?](#how-to-bypass-imperva) - [Which Imperva Bypass Tool Fixes Which Layer?](#which-imperva-bypass-tool-fixes-which-layer) - [Start With Residential Proxies and One Session per IP](#start-with-residential-proxies-and-one-session-per-ip) - [Use curl\_cffi for Imperva's TLS and HTTP/2 Checks](#use-curl-cffi-for-imperva-s-tls-and-http-2-checks) - [Try Nodriver](#try-nodriver) - [Try SeleniumBase: Selenium With UC Mode and CDP Mode](#try-seleniumbase-selenium-with-uc-mode-and-cdp-mode) - [Try Camoufox](#try-camoufox) - [Can You Reuse Imperva Cookies in curl\_cffi?](#can-you-reuse-imperva-cookies-in-curl-cffi) - [Browse Like a Visitor: Warm-Up, Pacing and Mouse Movement](#browse-like-a-visitor-warm-up-pacing-and-mouse-movement) - [Troubleshooting Common Imperva Blocks](#troubleshooting-common-imperva-blocks) - [Blocked on the First Request](#blocked-on-the-first-request) - [Blocked After a Few Pages](#blocked-after-a-few-pages) - [CAPTCHA Pages That Keep Coming Back](#captcha-pages-that-keep-coming-back) - [When a Working Scraper Starts Getting Blocked](#when-a-working-scraper-starts-getting-blocked) - [Bypass Imperva with ScrapFly](#bypass-imperva-with-scrapfly) - [Power your scraping with Scrapfly](#power-your-scraping-with-scrapfly) - [FAQ](#faq) - [Summary](#summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ blocking ### How to Bypass PerimeterX when Web Scraping in 2026 In this article we'll take a look at a popular anti scraping service PerimeterX. How does it detect web scrapers and bot... ](https://scrapfly.io/blog/posts/how-to-bypass-perimeterx-human-anti-scraping) [ ### How to Bypass Akamai when Web Scraping in 2026 In this article we'll take a look at a popular anti bot service Akamai Bot Manager. How does it detect web scrapers and ... ](https://scrapfly.io/blog/posts/how-to-bypass-akamai-anti-scraping) [ blocking ### How to Bypass Kasada Anti-Bot When Web Scraping in 2026 In this article, we'll take a look at the popular anti-bot service Kasada. How does it detect web scrapers and bots and ... ](https://scrapfly.io/blog/posts/how-to-bypass-kasada-anti-scraping-waf) ## Related Questions - [ Q What are Cloudflare Errors 1006, 1007, 1008? ](https://scrapfly.io/blog/answers/cloudflare-error-1006-1007-1008-access-denied) Bypass anti-bot protection automatically, **1,000 free credits** [Start Free](https://scrapfly.io/register)