[Blog](https://scrapfly.io/blog)   /  [AI Browser Agents - How to Use Them with Cloud Browsers](https://scrapfly.io/blog/posts/ai-browser-agents-how-to-use-them-with-cloud-browsers)   # AI Browser Agents - How to Use Them with Cloud Browsers

 by [Ziad Shamndy](https://scrapfly.io/blog/author/ziad) Jul 13, 2026 12 min read [  ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers "Share on LinkedIn") [  ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers&text=AI%20Browser%20Agents%20-%20How%20to%20Use%20Them%20with%20Cloud%20Browsers "Share on X") [  ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers "Share on Facebook")    

 

 

Summarize this article with

 [  ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers) [  ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers) [  ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers) [  ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers) [  ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fai-browser-agents-how-to-use-them-with-cloud-browsers) 



         

  **Browser API**Automate browsers in the cloud with full JavaScript rendering support.

 

 [ Learn More  ](https://scrapfly.io/products/cloud-browser-api) [  Docs ](https://scrapfly.io/docs/scrape-api/javascript-rendering) 

 

 

Most browser automation projects start with a fixed script and then fail when a target website changes one button label, adds a modal, or rearranges a checkout step. A fixed script still works for stable pages, but a fixed script often turns expensive when a data team must patch selectors every week across many targets.

This guide explains how to build an AI browser agent workflow that combines intelligent automation with cloud browser infrastructure. This guide covers when to use an AI browser agent, when to keep classic automation, architecture patterns, Browser Use integration, Stagehand integration, and practical next steps for production data pipelines.

## Key Takeaways

- Use AI browser agents for adaptive browser tasks, and use Scrapfly cloud browsers as the execution layer for reliability, anti-bot resilience, and scale.
- An AI browser agent can observe a page, decide a next action, and recover from moderate interface changes.
- Traditional Playwright or Puppeteer scripts remain the better option for deterministic, repeatable flows.
- Browser Use works well for Python teams that prefer natural-language task definitions.
- Stagehand works well for TypeScript teams that prefer structured actions and typed workflows.
- Scrapfly cloud browsers provide the browser infrastructure layer so Browser Use and Stagehand can run remotely with stable browser profiles.
- A practical architecture includes three layers: agent intelligence, browser infrastructure, and data extraction.

**Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime.







## What Are AI Browser Agents?

An AI browser agent is an autonomous system that can inspect browser state, reason over a goal, and execute multi-step actions to complete it. Unlike classic automation, it can choose new actions dynamically when page state changes.

Traditional browser automation relies on fixed scripts with predefined selectors, waits, and click paths. This approach is fast and predictable when interface structure stays stable, but becomes fragile when websites add dynamic overlays, change button hierarchy, or modify navigation flow.

AI browser agents adapt better in those changing environments because they use language-based reasoning and contextual page understanding. They can recover from small UI changes, unexpected pop-ups, and non-critical layout shifts without a full script rewrite.

Cloud browsers matter because these agents still need stable execution infrastructure. A cloud browser service handles browser provisioning, fingerprints, session continuity, and geographic routing while the AI framework focuses on planning and actions.

### When Not to Use AI Browser Agents

A balanced setup starts with tool selection, not hype. Use the following decision points before choosing an AI browser agent:

- Use Playwright or Puppeteer directly for simple and repeatable tasks such as fixed pagination loops or static form submission.
- Use traditional automation for cost-sensitive workloads where LLM calls for every interaction can create unnecessary runtime costs.
- Use deterministic scripts for high-reliability jobs where strict reproducibility matters more than adaptive behavior.
- Use AI browser agents when interfaces change frequently, pop-ups appear unpredictably, or natural-language task definitions reduce implementation time.

A practical team often uses both approaches in one pipeline. The next section shows the architecture that separates AI decision logic from browser infrastructure so each layer can evolve independently.

[Web Scraping with Playwright and PythonPlaywright is the new, big browser automation toolkit - can it be used for web scraping? In this introduction article, we'll take a look how can we use Playwright and Python to scrape dynamic websites.](https://scrapfly.io/blog/posts/web-scraping-with-playwright-and-python)



## Architecture: AI Framework + Cloud Browser Infrastructure + Data Extraction

A production AI browser automation stack is easiest to manage as three clear layers, each owning one responsibility and independently upgradeable. The AI framework layer handles task interpretation, planning, and action sequencing based on the current browser state. The browser infrastructure layer provides browser instances, remote sessions, anti-bot hardening, and connection endpoints for agent frameworks.

For cloud execution, [cloud browser infrastructure](https://scrapfly.io/docs/cloud-browser-api/getting-started) provides the infrastructure layer that Browser Use and Stagehand can connect to. Scrapfly does not replace Browser Use or Stagehand. Scrapfly powers Browser Use or Stagehand with remote browsers that are easier to operate at scale.

When raw page content is collected, the extraction layer converts raw content into typed objects, validates fields, and normalizes output shape across websites. Teams that need deeper extraction workflows can continue with [Scrapfly Extraction API documentation](https://scrapfly.io/docs/extraction-api/getting-started) after browser actions complete.

The Browser Use example below connects its Python agent directly to a Scrapfly CDP endpoint.



## Browser Use Integration with Scrapfly Cloud Browsers

[Browser Use](https://github.com/browser-use/browser-use) is an open-source Python framework for browser automation with agent behavior. Browser Use supports natural-language task prompts and integrates with modern LLM providers through standard Python tooling.

A Browser Use integration with Scrapfly follows a direct connection model. Browser Use owns reasoning and action planning. Scrapfly owns remote browser execution and anti-bot-aware infrastructure.

### Installation and Setup

Prepare a Python environment with Python 3.9+ and install required packages:

bash```bash
python -m venv .venv
source .venv/bin/activate
pip install browser-use langchain-openai python-dotenv
```



The commands above create an isolated environment and install Browser Use with a common OpenAI integration package. Use environment variables to store credentials before running agent code.

Create a `.env` file with API keys:

bash```bash
SCRAPFLY_API_KEY=sfp_your_api_key_here
OPENAI_API_KEY=sk_your_openai_key_here
```



The `.env` file keeps local development simple and avoids hardcoded secrets in source files. Create a [Scrapfly API key](https://scrapfly.io/dashboard) and an LLM API key before execution.

Use `browser_use` with Scrapfly CDP WebSocket URL:

python```python
import os
import asyncio
from dotenv import load_dotenv
from browser_use import Agent, Browser, ChatOpenAI


load_dotenv()

SCRAPFLY_API_KEY = os.environ["SCRAPFLY_API_KEY"]
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]

BROWSER_WS = (
    f"wss://browser.scrapfly.io?api_key={SCRAPFLY_API_KEY}"
    "&proxy_pool=datacenter&os=linux"
)

browser = Browser(cdp_url=BROWSER_WS)

llm = ChatOpenAI(model="gpt-4o-mini", api_key=OPENAI_API_KEY)


async def main() -> None:
    agent = Agent(
        task="Open https://httpbin.dev/anything and return the origin IP field value.",
        llm=llm,
        browser=browser,
    )
    result = await agent.run()
    print(result)


if __name__ == "__main__":
    asyncio.run(main())
```



The script above connects Browser Use to Scrapfly cloud browsers through `cdp_url`, then runs a simple agent task against `httpbin.dev`. The result usually includes the parsed IP response that confirms remote browser execution works end to end.

### Basic Agent Task on a Real Target

The next example uses `web-scraping.dev/products` to show a real scraping-friendly target. The task asks Browser Use to extract product names and prices as structured JSON.

python```python
import os
import asyncio
from dotenv import load_dotenv
from pydantic import BaseModel
from browser_use import Agent, Browser, ChatOpenAI

load_dotenv()

SCRAPFLY_API_KEY = os.environ["SCRAPFLY_API_KEY"]
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]

BROWSER_WS = (
    f"wss://browser.scrapfly.io?api_key={SCRAPFLY_API_KEY}"
    "&proxy_pool=datacenter&os=linux"
)


class Product(BaseModel):
    name: str
    price: str


class Products(BaseModel):
    products: list[Product]


browser = Browser(cdp_url=BROWSER_WS)
llm = ChatOpenAI(model="gpt-4o-mini", api_key=OPENAI_API_KEY)


async def scrape_products() -> None:
    task = (
        "Open https://web-scraping.dev/products. "
        "Extract the first 5 product names and prices. "
        "Return them as JSON conforming to the provided schema. "
        "Do NOT write to a file; return the data directly."
    )
    agent = Agent(
        task=task,
        llm=llm,
        browser=browser,
        output_model_schema=Products,
    )
    history = await agent.run()

    final = history.final_result()
    if final is None:
        raise RuntimeError("Agent did not produce a final result")

    parsed = Products.model_validate_json(final)
    print(parsed.model_dump_json(indent=2))


if __name__ == "__main__":
    asyncio.run(scrape_products())
```



The example targets a real scraping sandbox and keeps output schema explicit. Schema instructions improve reliability for downstream ingestion systems that expect predictable JSON fields.

For advanced options and framework-specific updates, use [Browser Use docs](https://scrapfly.io/docs/cloud-browser-api/browser-use).

Stagehand uses the same Scrapfly CDP endpoint pattern from TypeScript.

[How Javascript is Used to Block Web Scrapers? In-Depth GuideIntroduction to how javascript is used to detect web scrapers. What's in javascript fingerprint and how to correctly spoof it for web scraping.](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-javascript)



Scrapfly

#### Scale your web scraping effortlessly

Scrapfly handles proxies, browsers, and anti-bot bypass — so you can focus on data.

[Try Free →](https://scrapfly.io/register)## Stagehand Integration with Scrapfly Cloud Browsers

[Stagehand](https://github.com/browserbase/stagehand) is a TypeScript-first framework designed for structured browser actions and AI-assisted workflows. Stagehand uses a Playwright-based execution model, which makes Stagehand a strong fit for teams that already build on Playwright conventions.

The integration model is similar to Browser Use. Stagehand handles planning and actions in application code, while Scrapfly provides remote browser runtime through WebSocket connectivity.

### When to Use Stagehand vs Browser Use

Use framework choice based on language stack and control style:

- Choose Stagehand for TypeScript-heavy codebases, structured action primitives, and teams already operating in Playwright ecosystems.
- Choose Browser Use for Python-heavy codebases, natural-language prompt workflows, and teams using LangChain-based orchestration.

Both frameworks can use the same remote browser endpoint strategy. A shared browser infrastructure layer lets mixed-language teams standardize operations while preserving developer productivity.

For teams that need deeper Playwright context, see [Playwright-based](https://scrapfly.io/blog/posts/web-scraping-with-playwright-and-python) scraping patterns.

### Installation and Setup

Create a Node.js project and install dependencies:

bash```bash
mkdir stagehand-scrapfly-demo
cd stagehand-scrapfly-demo
npm init -y
npm pkg set type=module
npm install @browserbasehq/stagehand dotenv
```



The setup creates a minimal TypeScript/JavaScript environment for Stagehand execution. Store credentials in environment variables instead of source files.

Create a `.env` file:

bash```bash
SCRAPFLY_API_KEY=sfp_your_api_key_here
OPENAI_API_KEY=sk_your_openai_key_here
```



The environment file keeps local scripts clean and reusable across CI and developer machines.

Stagehand configuration with Scrapfly WebSocket URL:

javascript```javascript
import "dotenv/config";
import { Stagehand } from "@browserbasehq/stagehand";

const API_KEY = process.env.SCRAPFLY_API_KEY;

const BROWSER_WS = `wss://browser.scrapfly.io?api_key=${API_KEY}&proxy_pool=datacenter&os=linux`;

const stagehand = new Stagehand({
  env: "LOCAL",
  localBrowserLaunchOptions: {
    cdpUrl: BROWSER_WS,
  },
  model: "openai/gpt-4o-mini",
});

async function main() {
  await stagehand.init();

  const page = stagehand.context.pages()[0];
  await page.goto("https://httpbin.dev/anything", { waitUntil: "domcontentloaded" });

  const responseText = await page.locator("body").innerText();
  console.log(responseText);

  await stagehand.close();
}

main().catch(async (error) => {
  console.error(error);
  try {
    await stagehand.close();
  } catch {
    // no-op cleanup
  }
  process.exit(1);
});
```



The configuration uses `env: "LOCAL"` and points `localBrowserLaunchOptions.cdpUrl` to Scrapfly cloud browsers. The first connectivity test uses httpbin.dev/anything to confirm end-to-end remote execution.

### Deterministic Extraction After Stagehand Navigation

The next example uses Stagehand for remote browser navigation, then extracts products with fixed DOM selectors so the Python and TypeScript sections can be compared on the same data target.

javascript```javascript
import "dotenv/config";
import { Stagehand } from "@browserbasehq/stagehand";

const API_KEY = process.env.SCRAPFLY_API_KEY;
if (!API_KEY) {
  throw new Error("Missing SCRAPFLY_API_KEY environment variable");
}

const BROWSER_WS = `wss://browser.scrapfly.io?api_key=${API_KEY}&proxy_pool=datacenter&os=linux`;

const stagehand = new Stagehand({
  env: "LOCAL",
  localBrowserLaunchOptions: {
    cdpUrl: BROWSER_WS,
  },
  model: "openai/gpt-4o-mini",
});

async function run() {
  await stagehand.init();

  const page = stagehand.context.pages()[0];
  await page.goto("https://web-scraping.dev/products", { waitUntil: "networkidle" });

  // Structured extraction through page evaluation keeps output deterministic.
  const products = await page.evaluate(() => {
    const cards = Array.from(document.querySelectorAll(".product"));
    return cards.slice(0, 5).map((card) => {
      const name = card.querySelector(".product a")?.textContent?.trim() ?? "";
      const price = card.querySelector(".price")?.textContent?.trim() ?? "";
      return { name, price };
    });
  });

  console.log(JSON.stringify({ products }, null, 2));

  await stagehand.close();
}

run().catch(async (error) => {
  console.error(error);
  try {
    await stagehand.close();
  } catch {
    // no-op cleanup
  }
  process.exit(1);
});
```



The example keeps extraction deterministic with DOM selectors after Stagehand navigation and page handling. This hybrid pattern combines AI-assisted interaction with strict output structure for production pipelines.

For maintained integration examples and API changes, use [Stagehand docs](https://scrapfly.io/docs/cloud-browser-api/stagehand).

[Web Scraping With Cloud BrowsersIntroduction cloud browsers and their benefits and a step-by-step setup with self-hosted Selenium-grid cloud browsers.](https://scrapfly.io/blog/posts/web-scraping-with-cloud-browsers)



## FAQ

What is an AI browser agent in practical terms?An AI browser agent is a browser automation system that uses model-based reasoning to choose actions at runtime. A practical AI browser agent can navigate changing interfaces, recover from moderate UI shifts, and produce structured outputs from navigation goals.







Should a team replace Playwright scripts with AI browser agents?A team should not replace every script by default. A team should keep deterministic scripts for stable and repeatable flows, then add AI browser agents for unstable interfaces and workflow paths that change frequently.







Does cloud execution matter if local code already works?Cloud execution matters when a team needs stable browser runtime, anti-bot-aware browser profiles, and scalable remote sessions. Local execution can validate logic, while cloud execution usually improves operational reliability in production pipelines.







Which framework should a mixed-language team choose first?A mixed-language team should start with the framework that matches existing application ownership. Python-heavy teams usually move faster with Browser Use, while TypeScript-heavy teams usually move faster with Stagehand.









## Conclusion

AI browser automation works best when a team separates reasoning from runtime infrastructure. Browser Use and Stagehand solve the reasoning layer with different language preferences, while Scrapfly cloud browsers solve the remote execution layer for both frameworks.

A practical rollout path starts with one narrow task, one framework, and strict output schema. After the first workflow is stable, a team can expand to more targets, add session strategies, and standardize extraction contracts across projects.

Use [Scrapfly free tier](https://scrapfly.io/pricing) to start with a small pilot. Continue implementation with Cloud Browser documentation and use [Screenshot API](https://scrapfly.io/docs/screenshot-api/getting-started) for visual debugging when an agent workflow needs step-level verification.

[7 Best Cloud Browser APIs for Web Scraping in 2026Compare the best cloud and headless browser APIs for web scraping in 2026, from managed stealth browsers to self-hosted open-source engines.](https://scrapfly.io/blog/posts/best-cloud-browser-apis)



 

   [  Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents















 

  Table of Contents- [Key Takeaways](#key-takeaways)
- [What Are AI Browser Agents?](#what-are-ai-browser-agents)
- [When Not to Use AI Browser Agents](#when-not-to-use-ai-browser-agents)
- [Architecture: AI Framework + Cloud Browser Infrastructure + Data Extraction](#architecture-ai-framework-cloud-browser-infrastructure-data-extraction)
- [Browser Use Integration with Scrapfly Cloud Browsers](#browser-use-integration-with-scrapfly-cloud-browsers)
- [Installation and Setup](#installation-and-setup)
- [Basic Agent Task on a Real Target](#basic-agent-task-on-a-real-target)
- [Stagehand Integration with Scrapfly Cloud Browsers](#stagehand-integration-with-scrapfly-cloud-browsers)
- [When to Use Stagehand vs Browser Use](#when-to-use-stagehand-vs-browser-use)
- [Installation and Setup](#installation-and-setup)
- [Deterministic Extraction After Stagehand Navigation](#deterministic-extraction-after-stagehand-navigation)
- [FAQ](#faq)
- [Conclusion](#conclusion)
 
    Join the Newsletter  Get monthly web scraping insights 

 

  



Scale Your Web Scraping

Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits.

  No credit card required  1,000 free API credits  Anti-bot bypass included 

 [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) 

 Not ready? Get our newsletter instead. 

 

 ## Related Articles

 [     

 ai 

### Top LangChain Alternatives in 2026: Which One Should You Choose?

Explore the best LangChain alternatives in 2026 for building powerful AI applications. Compare features, performance, an...

 

 ](https://scrapfly.io/blog/posts/top-langchain-alternatives) [  

 ai 

### Guide to Understanding and Developing LLM Agents

Explore how LLM agents transform AI, from text generators into dynamic decision-makers with tools like LangChain for aut...

 

 ](https://scrapfly.io/blog/posts/practical-guide-to-llm-agents) [     

 python headless-browser 

### How to Create an AI Browser Agent for Free

Build two free AI browser agents using Browser-Use (Python) and Stagehand (TypeScript) with step-by-step code examples a...

 

 ](https://scrapfly.io/blog/posts/how-to-create-an-ai-browser-agent-for-free) 

  ## Related Questions

- [ Q How to run Playwright in Jupyter notebooks? ](https://scrapfly.io/blog/answers/playwright-in-ipython)
- [ Q How to use cURL in Python? ](https://scrapfly.io/blog/answers/how-to-use-curl-in-python)
 
  



   



 Scale your web scraping effortlessly, **1,000 free credits** [Start Free](https://scrapfly.io/register)