Scrapfly\_# Scrape any public site, drive any browser, power any agent. One API key. ## The web changes weekly. Headless browsers crash at scale. We absorb both so your scraper stays up. [ Get Free API Key ](https://scrapfly.io/register) [ Pricing ](https://scrapfly.io/pricing) 1,000 free credits. No credit card. [ ](https://www.capterra.com/p/212653/Scrapfly/) 4.9/5 from 236 reviews POST api.scrapfly.io/scrape { "success": true, "result": { "status\_code": 200, "duration": 1.24, "render\_js": true, "proxy": "residential/us", "browser": "chrome/148", "content": "<!doctype html>..." } } STATUS: one call, full stack FIG. 01 · ONE SCRAPE CALL, FULL STACK RESPONSE --- ## 15B+ requests / month ## 99.99% success rate ## 8PB+ data / month ## 30K+ enterprises --- PRODUCTS## One API key. Web scraping, browser automation, AI agents. Pick the tool. We handle the rest: proxies, browsers, retries, billing. - Web Scraping API - Cloud Browser - Screenshot API - Extraction API - Crawler API - AI Browser Agent - MCP Server Web Scraping API Fetch public URLs at production scale with managed proxy rotation and a JS rendering toggle. Clean HTML or markdown, stable JSON. $> `POST https://api.scrapfly.io/scrape` [ Web Scraping API ](https://scrapfly.io/products/web-scraping-api) [ Documentation ](https://scrapfly.io/docs/scrape-api/getting-started) Cloud Browser Drive a real managed Chromium over CDP. Full Playwright / Puppeteer compatibility. Hosted, scaled, production-grade. $> `wss://browser.scrapfly.io/cdp?key=...` [ Cloud Browser API ](https://scrapfly.io/products/cloud-browser-api) [ Documentation ](https://scrapfly.io/docs/cloud-browser-api/getting-started) Screenshot API Full-page, viewport, or element screenshots. PNG / JPEG / WebP. Ad-block, cookie-banner dismiss, custom viewport. $> `POST https://api.scrapfly.io/screenshot` [ Screenshot API ](https://scrapfly.io/products/screenshot-api) [ Documentation ](https://scrapfly.io/docs/screenshot-api/getting-started) Extraction API Turn HTML into typed data with a prompt or a JSON schema. LLM-powered, built-in templates for products, articles, reviews, jobs. $> `POST https://api.scrapfly.io/extraction` [ Extraction API ](https://scrapfly.io/products/extraction-api) [ Documentation ](https://scrapfly.io/docs/extraction-api/getting-started) Crawler API Traverse entire sites with depth limits, follow rules, rate control. Streams URLs as discovered; every page runs through the Web Scraping API. $> `POST https://api.scrapfly.io/crawler` [ Crawler API ](https://scrapfly.io/products/crawler-api) [ Documentation ](https://scrapfly.io/docs/crawler-api/getting-started) AI Browser Agent Managed Chromium tuned for autonomous agent loops. Browser Use, Stagehand, and Vibium compatible. Pay only for the actions that succeed. $> `POST https://api.scrapfly.io/agent` [ AI Browser Agent ](https://scrapfly.io/products/ai-browser-agent) [ Documentation ](https://scrapfly.io/products/ai-browser-agent) MCP Server Connect any MCP client (Claude, Cursor, Cline, Windsurf, ChatGPT) to scrape, screenshot, extract, and crawl with one API key. $> `mcp.scrapfly.io` [ MCP Server ](https://scrapfly.io/products/mcp-cloud) [ Documentation ](https://scrapfly.io/products/mcp-cloud) [Get Free API Key](https://scrapfly.io/register) --- // TARGET GUIDES## Pay on success. Failed requests are free. One parameter. No per-target configuration. Step-by-step guides for the targets developers ask about most. [ **Cloudflare guide** ](https://scrapfly.io/bypass/cloudflare) [ **Akamai guide** ](https://scrapfly.io/bypass/akamai) [ **DataDome guide** ](https://scrapfly.io/bypass/datadome) [ **Imperva guide** ](https://scrapfly.io/bypass/incapsula) [ **AWS WAF guide** ](https://scrapfly.io/bypass/aws-waf) [ **PerimeterX guide** ](https://scrapfly.io/bypass/perimeterx) [ **F5 guide** ](https://scrapfly.io/bypass/f5) [ **Kasada guide** ](https://scrapfly.io/bypass/kasada) [ Browse all target guides ](https://scrapfly.io/bypass) --- // LIVE## Observability included. Success rate, throughput, latency. SCRAPE LOG · /scrape tail -f SUCCESS RATE · last 60s up 47d 12h **99.99**% TARGETS **12,483** RPS **4,210** p50 LATENCY **1.12s** SUCCESS RATE **99.99%** FIG. 02 · OBSERVABILITY DASHBOARD, ILLUSTRATIVE ANIMATION --- ARCHITECTURE## Every layer, built in-house. Your API call. Our stack. From clean JSON all the way down to the exit IP. // LAYER 03 · API### What you call. One API key. Stable JSON envelope. Seven endpoints for scrapers, browsers, and AI agents. [Web Scraping](https://scrapfly.io/products/web-scraping-api) [Cloud Browser](https://scrapfly.io/products/cloud-browser-api) [Screenshot](https://scrapfly.io/products/screenshot-api) [Extraction](https://scrapfly.io/products/extraction-api) [Crawler](https://scrapfly.io/products/crawler-api) [AI Browser Agent](https://scrapfly.io/products/ai-browser-agent) [MCP Server](https://scrapfly.io/products/mcp-cloud) // LAYER 02 · ENGINES### How we run real Chrome. Two proprietary engines, auto-routed per target. Browser-grade HTTP and DOM behavior, engineered in-house. **HTTP/2** native **HTTP/3** QUIC **TLS 1.3** stack [ **Curlium** Proprietary HTTP engine with real-browser network behavior over HTTP/2 and QUIC. 10-50x faster than a browser. ](https://scrapfly.io/curlium) [ **Scrapium** Managed Chromium sharing Curlium's network stack, so you can alternate engines mid-session seamlessly. CDP, wait-for rules, custom scripts. ](https://scrapfly.io/scrapium) // LAYER 01 · FOUNDATION### Where we exit. Global proxy mesh across 190+ countries. Geo-aligned DNS, Accept-Language, and client-hints coherence on every request. Residential Datacenter TCP / UDP 190+ countries --- // INTEGRATIONS## Works with what you already use. Plug Scrapfly into your favorite AI agents, workflow automations, and first-class SDKs. ### No-code automation [ Zapier ](https://scrapfly.io/integration/zapier) [ Make ](https://scrapfly.io/integration/make) [ n8n ](https://scrapfly.io/integration/n8n) ### Agent skills [ OpenClaw ](https://scrapfly.io/integration/openclaw) ### LLM & RAG frameworks [ LlamaIndex ](https://scrapfly.io/integration/llamaindex) [ LangChain ](https://scrapfly.io/integration/langchain) [ CrewAI ](https://scrapfly.io/integration/crewai) ### First-class SDKs [ Python pip install scrapfly-sdk ](https://scrapfly.io/docs/sdk/python) [ TypeScript Node, Deno, Bun ](https://scrapfly.io/docs/sdk/typescript) [ Go go get github.com/scrapfly/go-scrapfly ](https://scrapfly.io/docs/sdk/golang) [ Rust cargo add scrapfly-sdk ](https://scrapfly.io/docs/sdk/rust) [ Scrapy Full-feature extension ](https://scrapfly.io/docs/sdk/scrapy) [ See all integrations ](https://scrapfly.io/integration) // WHY SCRAPFLY## Five reasons teams pick Scrapfly. What you get with us versus what you get with the rest of the category. // SCRAPFLY- Hard-target success rate98% on the hardest public targets, measured weekly by the [Scrapeway](https://scrapeway.com) benchmark. - Failed requestsFailed requests are free. You only pay for successful scrapes. - PricingCredits scale with the actual complexity of each target. - Product breadthOne API key covers [scraping](https://scrapfly.io/products/web-scraping-api), [browser automation](https://scrapfly.io/products/cloud-browser-api), [screenshots](https://scrapfly.io/products/screenshot-api), [extraction](https://scrapfly.io/products/extraction-api), [crawling](https://scrapfly.io/products/crawler-api), [agents](https://scrapfly.io/products/ai-browser-agent), and [MCP](https://scrapfly.io/products/mcp-cloud). - Stack ownershipOwns the entire stack: [in-house proxy mesh](https://scrapfly.io/products/proxy-saver), proprietary HTTP engine ([Curlium](https://scrapfly.io/curlium)), and managed browser ([Scrapium](https://scrapfly.io/scrapium)). // MOST SCRAPING APIS- Hard-target success rateOften well below 60% on the same targets. - Failed requestsFailed requests still consume credits. - PricingTiered pricing with premium-domain surcharges and rigid plans. - Product breadthMost cover scraping only. Browser or extraction usually means a second vendor. - Stack ownershipResells someone else's stack. [ See full vendor comparisons ](https://scrapfly.io/compare) --- // TESTIMONIALS## Trusted by 30,000+ enterprises. Teams pick Scrapfly after testing the alternatives. Here is why they stay. “It helped me automate a very difficult task that would have been nearly impossible to complete manually. The performance is consistent, the API responses are fast, and the documentation made integration simple. FA **Faten A.** AI Specialist [ Verified Capterra review ](https://www.capterra.com/p/212653/Scrapfly/) “I use Scrapfly as part of a larger AI research platform for web content extraction and data gathering. It was easy to integrate into our Node.js backend. MM **Megan M.** Principal Investigator, AI research platform [ Verified Capterra review ](https://www.capterra.com/p/212653/Scrapfly/) “Ran a 74k URL scraping job without major issues. Good value with the 200k credits on Discovery plan. GV **Gilbert V.** Growth [ Verified Capterra review ](https://www.capterra.com/p/212653/Scrapfly/) --- // PRICING## Try Scrapfly today. Same API key across every Scrapfly product. Pick a credit budget, every product shares the pool. No per-product lock-in, no surprise line items. FREE **1,000 credits on signup.** Every product, no credit card, no time limit. [ Get Free Credits ](https://scrapfly.io/register) // DISCOVERY **$30** /mo **$0.15** per 1k credits, prepaid **200,000** API credits / month - **5** concurrent requests - **1 week** log retention - No pay-as-you-go - Solo account - Basic Support [ Start Free ](https://scrapfly.io/register) // PRO **$100** /mo **$0.10** per 1k credits, prepaid **1,000,000** API credits / month - **20** concurrent requests - **2 weeks** log retention - Extra at **$3.50** / 10k - Solo account - Standard Support [ Start Free ](https://scrapfly.io/register) MOST POPULAR // STARTUP **$250** /mo **$0.10** per 1k credits, prepaid **2,500,000** API credits / month - **50** concurrent requests - **3 weeks** log retention - Extra at **$2.00** / 10k - Team management - Standard Support [ Start Free ](https://scrapfly.io/register) // ENTERPRISE **$500** /mo **$0.09** per 1k credits, prepaid **5,500,000** API credits / month - **100** concurrent requests - **4 weeks** log retention - Extra at **$1.20** / 10k - Team management - Premium Support [ Start Free ](https://scrapfly.io/register) // CUSTOM **Custom** **Negotiated** volume contract **Unlimited** credits negotiated - **Committed** concurrency - **Custom** log retention - **MSA, DPA, BAA, HIPAA** - Dedicated residential pools - Premium 24/7 support [ Talk to Sales ](mailto:sales@scrapfly.io?subject=Enterprise%20Plan%20Request) Full plan details on the [pricing page](https://scrapfly.io/pricing). No annual contract required, upgrade or downgrade anytime. --- // USE CASES## Built for every web-scraping workflow. From price monitoring to AI training corpora. One API, every vertical. [ E-commerce ](https://scrapfly.io/use-case/ecommerce-web-scraping) [ Finance ](https://scrapfly.io/use-case/finance-web-scraping) [ Travel ](https://scrapfly.io/use-case/travel-web-scraping) [ Real estate ](https://scrapfly.io/use-case/real-estate-web-scraping) [ Jobs ](https://scrapfly.io/use-case/jobs-web-scraping) [ SEO & SERP ](https://scrapfly.io/use-case/seo-and-serp-web-scraping) [ Social media ](https://scrapfly.io/use-case/social-media-web-scraping) [ News & media ](https://scrapfly.io/use-case/media-and-news-web-scraping) [ Lead gen ](https://scrapfly.io/use-case/web-scraping-leads) [ AI training ](https://scrapfly.io/use-case/ai-training-web-scraping) [ Compliance ](https://scrapfly.io/use-case/compliance-web-scraping) [ Fraud detection ](https://scrapfly.io/use-case/fraud-detection-web-scraping) [ Logistics ](https://scrapfly.io/use-case/logistics-web-scraping) [ See all ](https://scrapfly.io/use-case/web-scraping) --- // FROM THE BLOG## Web scraping guides, deep-dives, and updates. New tutorials and deep-dives every week. [ ](https://scrapfly.io/blog/posts/how-to-scrape-skyscanner) NEW python ### [How to Scrape Skyscanner Flight Prices in Python (2026)](https://scrapfly.io/blog/posts/how-to-scrape-skyscanner) Learn how to scrape Skyscanner flight results in Python by building deep-link URLs, rendering the search, and capturing the itinerary JSON the page loads. 24 min read [ ](https://scrapfly.io/blog/posts/how-to-scrape-airbnb) NEW python ### [How to Scrape Airbnb Listings and Prices (2026)](https://scrapfly.io/blog/posts/how-to-scrape-airbnb) Scrape Airbnb search results, listing details, fee-inclusive prices, guest reviews, and availability calendars with Python and Scrapfly. 22 min read [ ](https://scrapfly.io/blog/posts/how-to-scrape-imdb) NEW python ### [How to Scrape IMDb Data, Ratings, and Featured Reviews](https://scrapfly.io/blog/posts/how-to-scrape-imdb) Fetch protected IMDb pages through Scrapfly ASP, then parse JSON-LD and \_\_NEXT\_DATA\_\_ for movie metadata, ratings, featured reviews, search, charts, and person data. 16 min read [ ](https://scrapfly.io/blog/posts/css-selector-cheatsheet) data-parsing ### [Ultimate CSS Selector Cheatsheet for Web Scraping and HTML Parsing](https://scrapfly.io/blog/posts/css-selector-cheatsheet) CSS selectors is a powerful HTML querying protocol which is used by browsers to determine what HTML elements to style. It's also incredibly useful in HTML parsing when web scraping or processing HTML data, as the same queries can be used to select values as well. 25 min read [ ](https://scrapfly.io/blog/posts/parsing-html-with-chatgpt-code-interpreter) python ### [Crafting Web Scrapers using ChatGPT Code Interpreter is Easy](https://scrapfly.io/blog/posts/parsing-html-with-chatgpt-code-interpreter) The new chatgpt code intrepreter feature is an ideal assistant for crafting web scrapers. Here's how it can be used to help with HTML parsing. 12 min read [ ](https://scrapfly.io/blog/posts/how-to-scrape-twitter) python ### [How to Scrape Twitter (X.com) with Python in 2026](https://scrapfly.io/blog/posts/how-to-scrape-twitter) Scrape public Twitter (X.com) profiles and tweets with Python in 2026, and learn why guest tokens and doc\_ids break DIY scrapers. 19 min read [ ](https://scrapfly.io/blog/posts/guide-to-yahoo-finance-api) python ### [Yahoo Finance API: How to Get Yahoo Finance Data in Python (2026)](https://scrapfly.io/blog/posts/guide-to-yahoo-finance-api) Get Yahoo Finance data in Python with the yfinance library, the direct chart endpoint, and scraping, plus how to keep it running at scale. 14 min read [ ](https://scrapfly.io/blog/posts/jsonl-vs-json) data-parsing ### [JSONL vs JSON](https://scrapfly.io/blog/posts/jsonl-vs-json) Learn the differences between JSON and JSONLines, their use cases, and efficiency. Why JSONLines excels in web scraping and real-time processing 11 min read [ Browse the blog ](https://scrapfly.io/blog) --- FAQ## Frequently Asked Questions ### WHAT IS SCRAPFLY'S WEB SCRAPING API? A web scraping API is a service that fetches public URLs with managed proxy rotation and JavaScript rendering on every request, without you maintaining headless browsers or proxy stacks. Scrapfly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) delivers this with one HTTP call returning HTML, markdown, plain text, or structured JSON. You only pay for successful requests. ### WHAT'S SCRAPFLY'S SUCCESS RATE ON HARD-TO-SCRAPE SITES? 98% on the hardest public targets, measured weekly by the independent [Scrapeway](https://scrapeway.com) benchmark and tracked daily against production traffic. See the [success-rate benchmarks](https://scrapfly.io/bypass) for current numbers. ### HOW DOES SCRAPFLY HANDLE HARD-TO-SCRAPE SITES? Set `asp=true` on any request. Scrapfly analyzes the target and routes through the right engine: [Curlium](https://scrapfly.io/curlium) (high-performance real-browser HTTP layer) or [Scrapium](https://scrapfly.io/scrapium) (managed Chromium for JavaScript-heavy pages). Both share the same network stack, so sessions stay consistent from first request to last. One parameter, no per-target configuration. ### WHAT KIND OF PROXIES DOES SCRAPFLY OFFER? Two managed options. Use Scrapfly's built-in pools: datacenter (1 credit per request) or residential (25 credits per request, 190+ countries) with auto-rotation, IP cooling, and optional session stickiness. Or bring your own provider via [Proxy Saver](https://scrapfly.io/products/proxy-saver), which plugs Bright Data, Oxylabs, Webshare, SmartProxy, DataImpulse, or any SOCKS5 / HTTP proxy into the same Web Scraping API endpoint. Either way, smart routing, session management, and automatic retry sit on top. ### DOES SCRAPFLY WORK WITH AI AGENTS AND MCP CLIENTS? Yes. The [MCP Server](https://scrapfly.io/products/mcp-cloud) connects any MCP-compatible client (Claude, Cursor, Cline, Windsurf, ChatGPT, and others) to scrape, screenshot, extract, and crawl. The [AI Browser Agent](https://scrapfly.io/products/ai-browser-agent) supports Browser Use, Stagehand, and Vibium agent loops on managed Chromium. Native integrations also exist for LangChain, LlamaIndex, and CrewAI. ### WHAT OUTPUT FORMATS DOES SCRAPFLY SUPPORT? Multiple formats from a single API call: HTML (raw or sanitized), Markdown (with link / image stripping for LLM contexts), plain text, JSON (parsed or as the API envelope), screenshots in PNG / JPEG / WebP via the [Screenshot API](https://scrapfly.io/products/screenshot-api), structured JSON via the [Extraction API](https://scrapfly.io/products/extraction-api) using LLM prompts, CSS / XPath rules, or pre-trained models, and WARC archives for [crawl jobs](https://scrapfly.io/products/crawler-api). ### WHICH PROGRAMMING LANGUAGES DOES SCRAPFLY SUPPORT? Five official [SDKs](https://scrapfly.io/docs/sdk): [Python](https://scrapfly.io/docs/sdk/python), [TypeScript](https://scrapfly.io/docs/sdk/typescript) (Node, Deno, Bun), [Go](https://scrapfly.io/docs/sdk/golang), [Rust](https://scrapfly.io/docs/sdk/rust), and a [Scrapy](https://scrapfly.io/docs/sdk/scrapy) extension. The REST API works with any language via plain HTTP. The [CLI](https://scrapfly.io/docs/cli) handles shell pipelines and CI integration. ### HOW IS SCRAPFLY DIFFERENT FROM BRIGHT DATA, ZYTE, AND OXYLABS? Bright Data and Oxylabs are proxy-first platforms with scrapers built on top: strong on proxy infrastructure and managed datasets, heavier configuration per target. Zyte is built around Scrapy and managed services for high-protection enterprise targets. Scrapfly is single-endpoint: one API call with `asp=true` handles session management, proxy rotation, JavaScript rendering, and structured output with no per-target setup. You pay only for successful requests. See per-vendor comparisons on the [comparison hub](https://scrapfly.io/compare#grid). ### IS SCRAPFLY SOC 2 COMPLIANT? Yes. Scrapfly holds SOC 2 Type I, SOC 2 Type II, and SOC 3 certifications, plus ISO 27001, HIPAA, and GDPR attestations. A HIPAA Business Associate Agreement (BAA) is available to Custom-plan customers. See [Compliance](https://scrapfly.io/compliance) for the full list of attestations. ### DOES SCRAPFLY OFFER ENTERPRISE PLANS AND DEDICATED SUPPORT? Yes. The Enterprise tier is $500 / month (5.5M credits, 100 concurrent requests, premium support, team management). Custom plans above 5.5M credits / month include MSA, DPA, BAA, and HIPAA, dedicated residential proxy pools, committed concurrency, premium support, and custom log retention. Contact . See [pricing](https://scrapfly.io/pricing) for the full grid. ### WHAT HAPPENS IF A REQUEST FAILS? You are not charged. Failed requests do not consume API credits. The API returns a structured error response with retry guidance. Transient failures on demanding targets are automatically retried within the same request before returning an error. ### WHAT'S THE PRICING? Usage-based, priced per API credit. Free tier: 1,000 credits, no card, no time limit. Paid plans start at $30 / month (Discovery, 200k credits, 5 concurrent) and scale to $500 / month (Enterprise, 5.5M credits, 100 concurrent), with custom contracts above 5.5M credits / month. Credit cost per call scales with features used (1 credit for HTTP, +5 for JavaScript rendering or ASP, +25 for residential proxies, 60 for screenshot). Failed requests are free. See [pricing](https://scrapfly.io/pricing) for the full plan grid and credit matrix, or use the [cost estimator](https://scrapfly.io/cost_estimator) to size your project. --- // GET STARTED### Ship in a minute. Not a sprint. Free account, 1,000 credits, no credit card. One key, every product. [ Get Free API Key ](https://scrapfly.io/register) [See pricing](https://scrapfly.io/pricing)