[Blog](https://scrapfly.io/blog)   /  [headless-browser](https://scrapfly.io/blog/tag/headless-browser)   /  [Selenium Alternatives for Web Scraping: What to Use in 2026](https://scrapfly.io/blog/posts/best-selenium-alternatives-for-web-scraping)   # Selenium Alternatives for Web Scraping: What to Use in 2026

 by [Mayada Shaaban](https://scrapfly.io/blog/author/mayada-shaaban-90143e67) Nov 09, 2026 23 min read [\#headless-browser](https://scrapfly.io/blog/tag/headless-browser) [\#python](https://scrapfly.io/blog/tag/python) [\#selenium](https://scrapfly.io/blog/tag/selenium) 

 [  ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping "Share on LinkedIn") [  ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping&text=Selenium%20Alternatives%20for%20Web%20Scraping%3A%20What%20to%20Use%20in%202026 "Share on X") [  ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping "Share on Facebook")    

 

 

Summarize this article with

 [  ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping) [  ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping) [  ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping) [  ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping) [  ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-selenium-alternatives-for-web-scraping) 



         

One of the most difficult problems in replacing Selenium is that a new API doesn't change how our browser looks to websites! Both local Selenium and local Playwright failed the same two checks on Scrapfly's Automation Detection Tool: `navigator.webdriver=true` and `HeadlessChrome` in the user agent. So what should we replace Selenium with, and which part of our scraper does each alternative change?

In this guide, we'll take a look at Selenium alternatives by the job they do: browser interactions, self-run stealth browsers, parsing the response and crawling many pages. We'll see how they compare against each other with tested Python examples, and when proxies, fingerprint upkeep, CAPTCHAs and browser operations become the real job!



## Key Takeaways

- If the scraper still clicks and types, a library with auto-waiting locators removes most explicit wait code.
- If the data is already in the HTML or embedded JSON, an HTTP client and a parser can drop the browser entirely.
- For many server-rendered pages, a crawling framework replaces the Selenium loop with scheduling, retries and duplicate filtering.
- A stealth browser changes browser signals, but proxy reputation, challenges and upkeep stay with us - and the two tested here only ran on an unprotected page.
- Scrapfly's [Cloud Browser API](https://scrapfly.io/products/cloud-browser-api) runs Scrapium stealth Chromium over CDP, with a residential proxy option and opt-in CAPTCHA handling. Playwright flows can connect to it directly, while Selenium code has to be ported first, as there's no WebDriver endpoint.

**Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime.







## Which Selenium Alternative Fits Your Scraping Job?

Not all Selenium alternatives are made equal, even if they're all sold as Selenium replacements, as each one swaps out a different part of our scraper (be it the automation API, the browser or the crawl loop). So, we'd pick one by the job our scraper is doing - and when that job becomes running stealth browsers, proxies and challenge handling, Scrapfly can take it over.

So what does each one actually swap out? We'll be using this table to help visualize what each alternative replaces:

| Alternative | Job | Replaces | Keeps | Runs on | Returns |
|---|---|---|---|---|---|
| Playwright | Browser interactions | Selenium API | Selectors | Your machine | Page + locators |
| Camoufox / Nodriver | Self-run stealth | Browser + control | Extraction logic | Your machine | Page + elements |
| requests + BeautifulSoup | Data in HTML | Browser | Selectors | Your machine | Parsed fields |
| Scrapy | Multi-page crawls | Browser + crawl loop | Selectors | Your machine | Item feed |
| Scrapfly (our product) | Stealth at scale | Browser operations | Playwright flow | Scrapfly cloud | CDP, HTML or images |

There are 3 types of change among the open-source tools here: an API rewrite, a self-run stealth browser and removing the browser altogether. Meaning, Playwright still runs a normal browser and only swaps Selenium's calls. Camoufox is a patched Firefox with a Playwright-compatible wrapper, and Nodriver drives Chromium asynchronously over CDP. As for requests and Scrapy, they drop a browser we never needed in the first place!

Keep in mind our tests only used an unprotected page, so they don't tell us how a protected target would react.

For reference, these are the versions we tested with:

**Versions used in tests, October 22 and 30, 2026**

| Project | Tested release | Published | License |
|---|---|---|---|
| selenium (baseline) | 4.51.0 | Oct 9, 2026 | Apache-2.0 |
| playwright | 1.64.1 | Oct 22, 2026 | Apache-2.0 |
| requests | 2.34.2 | May 14, 2026 | Apache-2.0 |
| beautifulsoup4 | 4.15.0 | Jun 7, 2026 | MIT |
| scrapy | 2.19.0 | Sep 10, 2026 | BSD-3-Clause |
| camoufox (Python) | 0.5.8 | Oct 6, 2026 | MIT |
| nodriver | 0.50.99 | Oct 26, 2026 | AGPL-3.0 |

Note that the first five were tested on October 22 and the last two on October 30. Camoufox is licensed in two parts too: MIT for the Python wrapper and MPL-2.0 for the browser.



## Playwright: Rewrite Browser Interactions With Auto-Waiting Locators

Playwright is a cross-platform and cross-language web browser automation toolkit, which can drive Chromium, Firefox and WebKit. What makes it great for scraping is auto-waiting, as the [Playwright docs](https://playwright.dev/python/docs/actionability) say the library "auto-waits for all the relevant checks to pass and only then performs the requested action."

For example, our review scraper below only needs `reviews.first.wait_for()` instead of a `WebDriverWait` object with an expected condition. Pages that load their data late can still need an extra wait though. Here's how the usual Selenium calls map over:

| Selenium | Playwright for Python |
|---|---|
| `webdriver.Chrome(options=...)` | `p.chromium.launch()` |
| `driver.get(url)` | `page.goto(url)` |
| `WebDriverWait(...).until(...)` | Locator auto-wait |
| `find_elements(By.CSS_SELECTOR, s)` | `page.locator(s)` |
| `element.text` | `locator.inner_text()` |
| `driver.execute_script(js)` | `page.evaluate(js)` |
| `driver.save_screenshot(path)` | `page.screenshot(path=path)` |

Playwright for Python can be installed through `pip`, and then the `playwright install chromium` command will install the browser binaries - so there's no separate driver binary to match. Got a long Selenium flow to port? Then [playwright codegen](https://playwright.dev/python/docs/codegen) can help, as it records our clicks in a real browser window and writes them out as Python.

Let's take a look at a quick example where we wait for the reviews to load on a web-scraping.dev product page:

python

```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://web-scraping.dev/product/1")

    reviews = page.locator("#reviews .review")
    reviews.first.wait_for()  # replaces WebDriverWait + expected_conditions
    print(reviews.count())
    print(reviews.first.inner_text())

    browser.close()
```



Above, we used `reviews.first.wait_for()` to wait for the first review element to appear on the page, with no `time.sleep()` or wait object involved. If we run it, we'll get the same 5 reviews our Selenium scraper found:

text

```text
5
2022-07-22     

Absolutely delicious! The orange flavor is my favorite.
```



That being said, there are some negatives. While Playwright is using a real browser it's still possible to determine whether it's automated - local Playwright tripped the same two detected checks as local Selenium. Not only that, but we still run the browsers ourselves - and our Selenium Grid or WebDriver tooling won't carry over unchanged either.

For Node.js projects, Puppeteer is the JavaScript and TypeScript option, which we cover in our [Puppeteer alternatives](https://scrapfly.io/blog/posts/best-puppeteer-alternatives-for-web-scraping) guide.

[Playwright vs SeleniumExplore the key differences between Playwright vs Selenium in terms of performance, web scraping, and automation testing for modern web applications.](https://scrapfly.io/blog/posts/playwright-vs-selenium)

[Web Scraping With Playwright in 2026: A Python GuideScrape JavaScript-rendered pages with Playwright and Python in 2026: locators, waiting, request interception, resource blocking, and concurrency.](https://scrapfly.io/blog/posts/web-scraping-with-playwright-and-python)



## Camoufox and Nodriver: Run a Stealth Browser Yourself

Not every Selenium scraper fails on timing though - modern web browsers provide so much information about themselves that they can be easily identified and blocked. To prevent this, Camoufox gives us a patched Firefox with a Playwright-compatible Python API, and Nodriver gives us asynchronous Chromium control without WebDriver. Why not just Playwright? As we've seen, a plain Selenium-to-Playwright rewrite kept the same two detected signals!

[Camoufox](https://github.com/daijro/camoufox) is an open-source anti-detect Firefox build. Its Python wrapper exposes the familiar Playwright browser and page calls, but it's Firefox underneath, not Chromium over CDP. [Nodriver](https://github.com/ultrafunkamsterdam/nodriver), on the other hand, controls Chrome or Chromium over CDP without Selenium or a ChromeDriver binary. That being said, both of them still leave running the browsers and picking proxies up to us.

Camoufox's browser is still a beta though, and its README warns the project is under development and may not be suitable for stable production use. To try it out, we installed `camoufox==0.5.8` with `playwright==1.62.0` in a separate environment and ran `python -m camoufox fetch`, which got us browser build 156.0.1-beta.36 (though a later fetch can return a different one):

python

```python
from camoufox.sync_api import Camoufox

with Camoufox(headless=True) as browser:
    page = browser.new_page()
    response = page.goto(
        "https://web-scraping.dev/product/1",
        wait_until="domcontentloaded", timeout=60000,
    )
    text = page.locator("h3").first.inner_text(timeout=30000)
    assert response.status == 200, response.status
    assert text and "Chocolate" in text, repr(text)
    print(text)
```



text

```text
Box of Chocolate Candy
```



Nodriver is just as easy to set up, as all we need is `nodriver==0.50.99` and a Chrome or Chromium install. Note that the path below is for Linux, so swap in your own browser's location:

python

```python
import asyncio
import nodriver as uc

async def main():
    browser = await uc.start(
        headless=True, browser_executable_path="/usr/bin/chromium",
    )
    try:
        page = await browser.get("https://web-scraping.dev/product/1")
        element = await page.select("h3", timeout=30)
        text = element.text_all
        assert text and "Chocolate" in text, repr(text)
        print(text)
    finally:
        browser.stop()
        await asyncio.sleep(1)

asyncio.run(main())
```



text

```text
Box of Chocolate Candy
```



As you can see, both of them printed the product heading! Though, that's all these tests show, as we only navigated and extracted on an unprotected page. Meaning, there's no protected target, detection comparison or speed measurement behind them.

Why didn't the switch to Playwright help with detection then? Let's take a look at the 14 browser-side checks of Scrapfly's [Automation Detection Tool](https://scrapfly.io/web-scraping-tools/automation-detector) from our October 22 test. Keep in mind it's our own diagnostic page and not an independent benchmark - and we didn't run Camoufox or Nodriver on it:

| Setup | Detected | Suspicious | Safe |
|---|---|---|---|
| Selenium 4.51.0, Chrome 155 | 2 | 1 | 11 |
| Playwright 1.64.1, Chromium 156 | 2 | 3 | 9 |
| Our Cloud Browser, Playwright client | 0 | 1 | 13 |

In the table above, we can see that both local setups are identified by the same two signals: `navigator.webdriver=true` and `HeadlessChrome` in the user agent. On top of that, a missing `chrome.runtime` was suspicious in all three setups, and local Playwright also had zero plugins and MIME types.

With a fortified browser we can avoid instant identification, however our scraper can still be blocked as anti-bot services look at more than this diagnostic does. For example, TLS and HTTP2 fingerprints, IP reputation and behavior aren't tested here at all. So, even with zero detected checks we can't be sure we'll get through, or how often.

Unfortunately, these stealth browsers don't bring a trusted IP address with them, so we still need proxies of our own. Since web browsers change rapidly their patches can lag behind too, and our browser's identity has to match the proxy location as well - not to mention handling CAPTCHAs when they fail. If we'd rather keep WebDriver tooling, our [SeleniumBase guide](https://scrapfly.io/blog/posts/guide-to-seleniumbase-better-selenium) and [undetected-chromedriver guide](https://scrapfly.io/blog/posts/web-scraping-without-blocking-using-undetected-chromedriver) cover that path, though Selenium stays in the stack either way.

For more, see our [Nodriver tutorial](https://scrapfly.io/blog/posts/nodriver-tutorial) and [how anti-bot detection works](https://scrapfly.io/blog/posts/how-anti-bot-detection-works).

[Best Stealth Browsers for Web Scraping in 2026A neutral, benchmark-literate ranking of the best open-source stealth browsers for web scraping in 2026, with the success, speed, and maintenance tradeoffs.](https://scrapfly.io/blog/posts/best-stealth-browsers)



## Requests and BeautifulSoup: Parse Data Already in the HTML

When it comes to replacing Selenium we have two extreme ends: plow through the page with browser automation or skip the browser altogether. Common idiom for the latter is to request the page with an HTTP client like requests and parse the data with beautifulsoup - as long as the HTML, a hidden JSON variable or a data API already contains what we need. We have to get our hands dirty but our scraper will be significantly faster - in our five-page timing test requests and beautifulsoup took 1.33 seconds while Selenium took 3.06 seconds!

The easiest way to determine whether we need a browser at all is to employ a simple test:

1. Check page source (right click on the page, not the inspector) and look for our data (e.g. ctrl+f).
2. Look for the data in `<script>` HTML tags - it could be a JSON object or a framework state variable.
3. Open the Network tab of the browser developer tools and see if there's a data request we could call directly.

Let's take a look at how this works on the same product page using requests and beautifulsoup:

python

```python
import json
import requests
from bs4 import BeautifulSoup

response = requests.get("https://web-scraping.dev/product/1", timeout=10)
response.raise_for_status()
soup = BeautifulSoup(response.text, "html.parser")

print(len(soup.select("#reviews .review")))

reviews = json.loads(soup.select_one("#reviews-data").get_text())
print(len(reviews))
print(reviews[0])
```



If we run it, we can see that the raw HTML has no review nodes at all. However, the data is still there - the `#reviews-data` script holds all 5 initial reviews as JSON!

text

```text
0
5
{'date': '2022-07-22', 'id': 'chocolate-candy-box-1', 'rating': 5, 'text': 'Absolutely delicious! The orange flavor is my favorite.'}
```



On the bright side, many CSS selectors we used with Selenium can be moved to `soup.select()` as they are, though Selenium objects and interaction code have to be rewritten.

That being said, the main disadvantage of this approach is that there's no javascript execution or clicking. For example, the Load More button on this page pulls more reviews through a separate request, meaning the embedded JSON only has the first 5 reviews and to get the rest we have to replicate that request ourselves. Another downside is that protected websites can block plain HTTP clients altogether - that's a blocking problem rather than a parsing one, so beautifulsoup can't help us there. For more on these see [how to scrape hidden web data](https://scrapfly.io/blog/posts/how-to-scrape-hidden-web-data) and [how to capture background requests in Selenium](https://scrapfly.io/blog/answers/how-to-capture-xhr-requests-selenium).

[How to Parse Web Data with Python and BeautifulsoupBeautifulsoup is one the most popular libraries in web scraping. In this tutorial, we'll take a hand-on overview of how to use it, what is it good for and explore a real -life web scraping example.](https://scrapfly.io/blog/posts/web-scraping-with-python-beautifulsoup)



## Scrapy: Schedule Multi-Page Crawls Without a Browser

Selenium works fine for a handful of pages, however when scaling to hundreds of server-rendered pages reinventing the wheel in our own loop can be a suboptimal and painful experience. For this it might be worth taking a look at Scrapy, which implements a lot of things that otherwise would be difficult to implement by hand, such as request scheduling, concurrency, retry logic, duplicate URL filtering and data exports.

Scrapy is a free and open source (BSD-3-Clause) framework and it comes with all of these batteries included - all we have to provide are the parsing callbacks and the item fields. Let's take a look at a spider for the web-scraping.dev product listing:

python

```python
import scrapy


class ProductSpider(scrapy.Spider):
    name = "products"

    async def start(self):
        # a plain Request keeps duplicate filtering on (start_urls doesn't)
        yield scrapy.Request("https://web-scraping.dev/products?page=1")

    def parse(self, response):
        for product in response.css(".products .product"):
            yield {
                "title": product.css("h3 a::text").get(),
                "price": product.css(".price::text").get(),
            }
        yield from response.follow_all(css=".paging a", callback=self.parse)
```



Running it with `scrapy runspider spider.py -O products.json` fetched 5 pages and wrote 25 items, as the stats at the end of the log show:

text

```text
 'downloader/request_count': 5,
 'dupefilter/filtered': 25,
 'elapsed_time_seconds': 1.5953276249929331,
 'item_scraped_count': 25,
```



Why the `async def start()` instead of `start_urls`? When we used `start_urls`, the spider made 6 requests and scraped 30 items - page 1 got crawled twice!

That's because the default start requests have `dont_filter=True` (see the [Scrapy spiders docs](https://docs.scrapy.org/en/latest/topics/spiders.html)), while a plain `scrapy.Request` yielded from `async def start()` keeps duplicate filtering on. Note that async `start()` is available from Scrapy 2.13.

Unfortunately, Scrapy has no built-in JavaScript renderer, and add-ons for that bring browser operations back. Scrapy - being a full framework - is also a bit of an overkill for one or two pages where a short HTTP script will do. Plain HTTP spiders can still get blocked as well, for more on that see [why Scrapy spiders get blocked](https://scrapfly.io/blog/posts/why-scrapy-spider-gets-blocked) and [Scrapy with Playwright](https://scrapfly.io/blog/posts/how-to-use-scrapy-with-playwright).

[Web Scraping With Scrapy: The Complete Guide in 2026Build and run a Scrapy 2.18 project with async start(), pagination, selector tests, item validation, pipelines, and JSON Lines export.](https://scrapfly.io/blog/posts/web-scraping-with-scrapy)



## How Do You Choose a Selenium Alternative for Your Scraper?

We've covered four Selenium alternatives: Playwright, Camoufox and Nodriver, requests with beautifulsoup and Scrapy - so which one should we stick with?

To put it shortly - it all depends on what's wrong with our Selenium scraper:

| Symptom | Replace | Keep | Use |
|---|---|---|---|
| Flaky explicit waits | Automation library | Selectors | Playwright |
| Data in HTML or JSON | Browser | Parsing logic | requests + BeautifulSoup |
| Many paginated pages | Browser + crawl loop | Item fields | Scrapy |
| Browser signals detected | Browser + control | Extraction logic | Camoufox / Nodriver |
| IP blocks or challenge upkeep | Access infrastructure | Parsing / flow logic | Scrapfly |
| Existing Grid or WebDriver need | Nothing by default | Existing stack | Keep Selenium |

So, we only need to replace the part of our scraper that's causing the problem. Then, to make sure nothing broke, we can scrape the same URLs before and after the switch and compare the results.

Same goes for blocking - it's rarely the client library's fault, so check the response, the browser signals and the network path first. Self-run stealth browsers like Camoufox fix the browser signals, however solutions to proxies, browser builds and challenges still have to be implemented by each developer on their own. When that becomes the bottleneck Scrapfly is a better fit than another rewrite. For testing rather than scraping see our [browser automation tools](https://scrapfly.io/blog/posts/best-browser-automation-tools) roundup.



Scrapfly

#### Need a cloud browser for scraping?

Run headless browsers at scale with Scrapfly Cloud Browser — no infrastructure to manage.

[Try Free →](https://scrapfly.io/register)## Is Selenium Slower Than Playwright and BeautifulSoup?

So when it comes to speed: is Selenium slow because of its WebDriver commands? Not exactly. Local WebDriver commands are cheap and only take a few milliseconds. We can easily send hundreds of commands but if we're loading pages in a full browser most of our time will be taken up by the browser itself. So, instead, it's worth checking which of these two is slowing our scraper down before we migrate!

You'll often read that every WebDriver command takes 50-200 ms. So, if we make 200 calls of each operation with local ChromeDriver on an already loaded page, we should end up spending 10+ seconds per operation:

| Operation (200 calls) | Median | p95 |
|---|---|---|
| Selenium `find_element` | 1.10 ms | 1.53 ms |
| Selenium `execute_script` | 0.73 ms | 0.89 ms |
| Selenium find + `.text` | 2.49 ms | 3.09 ms |
| Playwright `locator.count()` | 0.38 ms | 0.91 ms |
| Playwright `evaluate` | 0.25 ms | 0.33 ms |
| Playwright `inner_text` | 0.46 ms | 0.80 ms |

Instead, every Selenium call took a millisecond or two and Playwright calls took even less - nowhere near 50-200 ms!

For local tests like these a common issue is that the browsers are not always the same browsers - ours were Chrome 155 for Selenium and Chromium 156 for Playwright.

Driver version can frequently be a source of issues as well. In our case, ChromeDriver 154 was picked up from PATH while Chrome was on 155 - we got a version mismatch warning but all tests completed. It's something worth keeping an eye on.

Finally, since remote Grid or remote WebDriver setups send every command over the network, their command costs can vary greatly by each setup - these local numbers can't predict them.

So what about the whole scrape? Counting 25 products on our five listing pages took 1.33 seconds with requests and beautifulsoup and 3.06 seconds with one headless Selenium Chrome, including launch and quit (medians of three runs).

Slow Selenium scrapers are nothing new. In a [2020 Stack Overflow question](https://stackoverflow.com/questions/59826577/making-webscraping-with-selenium-faster-or-faster-alternatives), Ton's form-and-AJAX scraper took "+- 12 seconds", and CEH replied: "If you want more speed, use `requests`." While Playwright's auto-waiting changes how we write interaction code, generally skipping the browser and using HTML parsing libraries (such as beautifulsoup in Python) will result in faster scraper code when the response already has the data. For waits that are still needed, see [how to wait for a page to load in Selenium](https://scrapfly.io/blog/answers/how-to-wait-for-page-to-load-in-selenium).



## Is Selenium Still Relevant in 2026?

Yes! Selenium is one of the first big automation clients created for automating web-site testing and it's still a very popular package that is implemented in multiple languages. Meaning it has a huge community, a lot of existing test code and Grid setups built around it - and a blocked scraper isn't proof that Selenium is obsolete.

It's still actively released too - 4.51.0 came out on October 9 and 4.52.0 on October 29, 2026 (see [selenium on PyPI](https://pypi.org/project/selenium/)). On top of that, since 4.6 Selenium comes with [Selenium Manager](https://www.selenium.dev/documentation/selenium_manager/) which downloads the right driver for us when there isn't one.

Selenium also supports more than the older request-response webdriver protocol now. Using [WebDriver BiDi](https://www.selenium.dev/documentation/webdriver/bidi/) we can listen for browser events (like network requests or console logs) instead of only sending commands. Need more browsers? Selenium Grid can run them on multiple machines.

For more, see [web scraping with Selenium and Python](https://scrapfly.io/blog/posts/web-scraping-with-selenium-and-python), [Selenium Grid for web scraping](https://scrapfly.io/blog/posts/intro-to-web-scraping-using-selenium-grid) and [proxies with Selenium](https://scrapfly.io/blog/posts/how-to-use-proxies-with-selenium).



## When Stealth Browsers Hit Their Limits: Scrapfly Cloud Browser

Running stealth browsers ourselves can be a difficult, time-consuming process: there are a lot of moving parts and variables - so many things that can go wrong. For this at Scrapfly (that's us!) we're offering to do the heavy lifting with Cloud Browser - it runs our Scrapium stealth Chromium so we can keep our Playwright code without running the browser fleet ourselves.

So why not just run a stealth browser ourselves?

Well, we can get a fingerprint check to pass on our own machine but that doesn't solve everything. We'd still need to find proxies, keep the fingerprint patches up to date, deal with CAPTCHAs and scale it all up by ourselves.

Scrapfly's [Cloud Browser API](https://scrapfly.io/products/cloud-browser-api) runs every session on [Scrapium](https://scrapfly.io/scrapium), Scrapfly's patched Chromium fork, over CDP. Scrapfly maintains the browser and its fingerprint patches, so all we have to change in our code is `launch()` to `connect_over_cdp()`.

Unfortunately, Selenium can't connect here directly (see the [Selenium integration docs](https://scrapfly.io/docs/cloud-browser-api/selenium)), so if we're coming from Selenium we'll have to port our code to Playwright first.

Let's quickly take a look at how we can connect our Playwright example from earlier to Cloud Browser using the [Python SDK](https://scrapfly.io/docs/sdk/python) (we'll need `scrapfly-sdk` and `playwright` installed and our key in the `SCRAPFLY_KEY` variable):

python

```python
import os
from playwright.sync_api import sync_playwright
from scrapfly import ScrapflyClient, BrowserConfig

client = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"])
ws_url = client.cloud_browser(BrowserConfig(
    proxy_pool="datacenter",
    target_url="https://web-scraping.dev/product/1",
))

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(ws_url)  # instead of p.chromium.launch()
    try:
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()
        page.goto("https://web-scraping.dev/product/1")

        reviews = page.locator("#reviews .review")
        reviews.first.wait_for()
        print(reviews.count())
        print(page.evaluate("navigator.webdriver"))
    finally:
        browser.close()
```



Here, `client.cloud_browser()` builds the connection URL from our `BrowserConfig` and the only real change to our Playwright flow is `connect_over_cdp()` with `browser.contexts[0]`. If we run it, we get the same 5 reviews - and this time `navigator.webdriver` is turned off:

text

```text
5
False
```



The whole flow took 11.28 seconds from connection to extraction (browser allocation included), and there was no `HeadlessChrome` in the user agent either!

If we check it against the diagnostic table from before, we get the same 0 detected, 1 suspicious and 13 safe result. Keep in mind it's just a test page!

Blocked datacenter IPs? Switch to [residential routing](https://scrapfly.io/docs/cloud-browser-api/proxy). CAPTCHAs? We can set `solve_captcha=true` and Scrapfly will try to solve them for us (for more on that see the [CAPTCHA solver docs](https://scrapfly.io/docs/cloud-browser-api/captcha-solver)). Neither is a sure thing though!

Unfortunately, there are some downsides too. It only works with Chromium and connects through CDP, which Playwright doesn't support as well as its own protocol. Also, the browser isn't on our machine so each command we send makes a round trip over the network.

For longer flows there's [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume) - just don't forget to close the browser when done, as it keeps billing!

What if we only need the page and not a browser to drive? For that, the [Web Scraping API](https://scrapfly.io/products/web-scraping-api) returns page content in a single request - all we need is `render_js=True` and `wait_for_selector` for rendered content:

python

```python
import os
from scrapfly import ScrapflyClient, ScrapeConfig

client = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"])

result = client.scrape(ScrapeConfig(
    url="https://web-scraping.dev/product/1",
    render_js=True,
    wait_for_selector="#reviews .review",
))

print(result.upstream_status_code)
print(len(result.selector.css("#reviews .review")))
```



Note that `result.selector` needs `parsel`, so we'll install it with `pip install scrapfly-sdk parsel`. If we run this, we get the upstream status code and the same 5 review nodes:

text

```text
200
5
```



This test page isn't protected so we didn't need the Unblocker here. For protected pages there's [Unblocker](https://scrapfly.io/products/unblocker) (`unblocker=True`, previously ASP).

With `format="markdown"` we got 4,580 characters of Markdown back, at 6 credits per rendered request.

For clicking and scrolling there's [js\_scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario), but branching flows are better left to Cloud Browser.

And for screenshot-only Selenium jobs there's the [Screenshot API](https://scrapfly.io/products/screenshot-api) - our full-page PNG was 1,238,146 bytes. For the SDK recipe see the [Screenshot API docs](https://scrapfly.io/docs/screenshot-api/getting-started), and for the local version see [how to take a screenshot with Selenium](https://scrapfly.io/blog/answers/how-to-take-screenshot-with-selenium).

Scrapfly has TypeScript, Go and Rust SDKs as well - so give your existing Playwright flow a shot with [Scrapfly's free plan](https://scrapfly.io/register)!



### Web Scraping API

Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in.



[Try Web Scraping API](https://scrapfly.io/docs/scrape-api/getting-started)



## FAQ

Is Playwright Replacing Selenium in 2026?Not entirely. Playwright's auto-waiting makes it a great fit for new scrapers, however Selenium is still actively released and existing WebDriver and Grid projects can keep using it - for more see our [Selenium web scraping](https://scrapfly.io/blog/posts/web-scraping-with-selenium-and-python) guide.







Can a Stealth Browser Stop Selenium Blocking?Partially. Camoufox and Nodriver change what the browser looks like, however they can't do much about a flagged IP address or a CAPTCHA - for that see our [IP address blocking](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-ip-addresses) guide.







Is There a Python Version of Puppeteer?Kind of. Pyppeteer is the unofficial Python port of Puppeteer, however its [README](https://github.com/pyppeteer/pyppeteer) says the repository is unmaintained and points users to Playwright for Python instead. Puppeteer itself is JavaScript-only, so we went with Playwright for this guide.







Which Selenium Alternative Works for C#?Playwright, as it has an official [Playwright for .NET](https://playwright.dev/dotnet/) binding. Requests, BeautifulSoup and Scrapy are Python-only though.









## Summary

In this overview article, we took a glance at the most popular Selenium alternatives in the context of web-scraping. We took a look at Playwright and its auto-waiting locators, stealth browsers like Camoufox and Nodriver, the classic requests and BeautifulSoup combo for pages that don't need a browser at all and Scrapy for crawling many pages.

Finally, we covered a few common challenges when it comes to proxies and keeping stealth browsers up to date, and how Scrapfly's Cloud Browser is designed to take care of them for you!

Selenium, while older, is still a great generic solution for existing test code and Grid setups - but for new scrapers give these alternatives a shot!



## How Do Other Scraping APIs and Hosted Browsers Compare?

For side-by-side looks at other hosted browsers and scraping APIs, see our comparison pages:

- [Browserless alternative](https://scrapfly.io/compare/browserless-alternative)
- [Browserbase alternative](https://scrapfly.io/compare/browserbase-alternative)
- [Firecrawl alternative](https://scrapfly.io/compare/firecrawl-alternative)
- [ZenRows alternative](https://scrapfly.io/compare/zenrows-alternative)
- [ScrapeGraphAI alternative](https://scrapfly.io/compare/scrapegraphai-alternative)
- [Apify alternative](https://scrapfly.io/compare/apify-alternative)
- [ScraperAPI alternative](https://scrapfly.io/compare/scraperapi-alternative)
- [ScrapeOps alternative](https://scrapfly.io/compare/scrapeops-alternative)



Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect:

- Do not scrape at rates that could damage the website.
- Do not scrape data that's not available publicly.
- Do not store PII of EU citizens protected by GDPR.
- Do not repurpose *entire* public datasets which can be illegal in some countries.

Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer.

 

   [  Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents















 

  Table of Contents

- [Key Takeaways](#key-takeaways)
- [Which Selenium Alternative Fits Your Scraping Job?](#which-selenium-alternative-fits-your-scraping-job)
- [Playwright: Rewrite Browser Interactions With Auto-Waiting Locators](#playwright-rewrite-browser-interactions-with-auto-waiting-locators)
- [Camoufox and Nodriver: Run a Stealth Browser Yourself](#camoufox-and-nodriver-run-a-stealth-browser-yourself)
- [Requests and BeautifulSoup: Parse Data Already in the HTML](#requests-and-beautifulsoup-parse-data-already-in-the-html)
- [Scrapy: Schedule Multi-Page Crawls Without a Browser](#scrapy-schedule-multi-page-crawls-without-a-browser)
- [How Do You Choose a Selenium Alternative for Your Scraper?](#how-do-you-choose-a-selenium-alternative-for-your-scraper)
- [Is Selenium Slower Than Playwright and BeautifulSoup?](#is-selenium-slower-than-playwright-and-beautifulsoup)
- [Is Selenium Still Relevant in 2026?](#is-selenium-still-relevant-in-2026)
- [When Stealth Browsers Hit Their Limits: Scrapfly Cloud Browser](#when-stealth-browsers-hit-their-limits-scrapfly-cloud-browser)
- [FAQ](#faq)
- [Summary](#summary)
- [How Do Other Scraping APIs and Hosted Browsers Compare?](#how-do-other-scraping-apis-and-hosted-browsers-compare)
 
    Join the Newsletter  Get monthly web scraping insights 

 

  



Scale Your Web Scraping

Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits.

  No credit card required  1,000 free API credits  Anti-bot bypass included 

 [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) 

 Not ready? Get our newsletter instead. 

 

 ## Related Articles

 [     

 python data-parsing 

### How to Parse Web Data with Python and Beautifulsoup

Beautifulsoup is one the most popular libraries in web scraping. In this tutorial, we'll take a hand-on overview of how ...

 

 ](https://scrapfly.io/blog/posts/web-scraping-with-python-beautifulsoup) [     

 headless-browser puppeteer 

### 4 Best Puppeteer Alternatives for Web Scraping in 2026

A job-by-job guide to replacing Puppeteer for web scraping, covering library rewrites, crawl frameworks, no-browser rout...

 

 ](https://scrapfly.io/blog/posts/best-puppeteer-alternatives-for-web-scraping) [     

 api ai 

### 7 Best Lead Scraping Tools in 2026

Ranked list of 7 lead scraping tools that pull fresh data from the live web, from no-code extensions to production APIs,...

 

 ](https://scrapfly.io/blog/posts/best-lead-scraping-tools) 

  ## Related Questions

- [ Q How to scrape tables with BeautifulSoup? ](https://scrapfly.io/blog/answers/how-to-scrape-tables-with-beautifulsoup)
- [ Q What's the difference between Web Scraping and Crawling? ](https://scrapfly.io/blog/answers/whats-the-difference-between-scraping-and-crawling)
- [ Q Scrapy vs BeautifulSoup: Which Should You Use? ](https://scrapfly.io/blog/answers/scrapy-vs-beautifulsoup)
- [ Q How to save and load cookies in Selenium? ](https://scrapfly.io/blog/answers/how-to-save-and-load-cookies-in-selenium)
 
  



   



 Run headless browsers at scale, **1,000 free credits** [Start Free](https://scrapfly.io/register)