# ScrapingBee to Scrapfly Migration Guide Complete parameter mapping and code examples for migrating from ScrapingBee to Scrapfly. Most teams complete migration in under 2 hours with zero downtime. [ Start For Free - 1,000 Credits ](https://scrapfly.io/register) [ Back to ScrapingBee Alternative Page ](https://scrapfly.io/compare/scrapingbee-alternative) ## [Complete Parameter Mapping](#parameter-mapping) ScrapingBee and Scrapfly share similar concepts with different parameter names. This table shows exact mappings for all features. | ScrapingBee Parameter | Scrapfly Parameter | Notes | |---|---|---| | `api_key` | [`key`](https://scrapfly.io/docs/scrape-api/getting-started) | API authentication key | | `url` | [`url`](https://scrapfly.io/docs/scrape-api/getting-started#url) | Target URL to scrape (same) | | `render_js` | [`render_js`](https://scrapfly.io/docs/scrape-api/javascript-rendering) | JavaScript rendering (same name). Note: ScrapingBee defaults to `true`, Scrapfly defaults to `false` | | `stealth_proxy` | [`asp`](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) | Anti-bot bypass (Anti-Scraping Protection). 75 credits → 30+ credits | | `premium_proxy` | [`proxy_pool`](https://scrapfly.io/docs/scrape-api/proxy#proxy-pool) | Use `public_residential_pool` for residential proxies | | `country_code` | [`country`](https://scrapfly.io/docs/scrape-api/proxy#country) | 2-letter ISO country code (e.g., "us", "gb") | | `wait` | [`rendering_wait`](https://scrapfly.io/docs/scrape-api/javascript-rendering#rendering-wait) | Wait time in milliseconds before returning response (0-35000 → 1000-25000) | | `wait_for` | [`wait_for_selector`](https://scrapfly.io/docs/scrape-api/javascript-rendering#wait-for-selector) | CSS selector to wait for before rendering complete | | `wait_browser` | [`rendering_stage`](https://scrapfly.io/docs/scrape-api/javascript-rendering#rendering-stage) | Values: `domcontentloaded`, `load`, `complete` | | `js_scenario` | [`js_scenario`](https://scrapfly.io/docs/scrape-api/javascript-scenario) | JavaScript scenario for browser automation (similar JSON format) | | `session_id` | [`session`](https://scrapfly.io/docs/scrape-api/session) | Session name for persistent cookies/state | | `cookies` | [`cookies`](https://scrapfly.io/docs/scrape-api/session#cookies) | Custom cookies to send (JSON format) | | `forward_headers` + headers | [`headers`](https://scrapfly.io/docs/scrape-api/proxy#custom-headers) | Custom HTTP headers (JSON format, no prefix needed) | | `extract_rules` | [`extraction_template`](https://scrapfly.io/docs/extraction-api/rules-and-template) | Use Scrapfly's Extraction API for structured data | | `ai_query` / `ai_extract_rules` | [`extraction_prompt`](https://scrapfly.io/docs/extraction-api/llm-prompt) | AI-powered data extraction with natural language | | `screenshot` | [`screenshots`](https://scrapfly.io/docs/screenshot-api/getting-started) | Capture screenshots (use Screenshot API or screenshots param) | | `screenshot_full_page` | `screenshots[main]=fullpage` | Capture full page screenshot | | `screenshot_selector` | `screenshots[main]=selector:CSS_SELECTOR` | Screenshot a specific element | | `device` | [`os`](https://scrapfly.io/docs/scrape-api/javascript-rendering#os) | Operating system for browser fingerprint (desktop/mobile) | | `window_width` / `window_height` | `resolution` | Viewport dimensions (e.g., `1920x1080`) | | `json_response` | [`format`](https://scrapfly.io/docs/scrape-api/getting-started#api_param_format) | Response format: `raw`, `markdown`, `text`, `clean_html`, `json` | | `return_page_source` | (default behavior) | Scrapfly returns HTML by default | | `return_page_markdown` | `format=markdown` | Return page content in Markdown format | | `return_page_text` | `format=text` | Return plain text content | | `block_ads` | N/A | Not directly available; use `format_options` for content filtering | | `block_resources` | N/A | Scrapfly optimizes resource loading automatically | | `timeout` | [`timeout`](https://scrapfly.io/docs/scrape-api/getting-started#timeout) | Request timeout in milliseconds | | `transparent_status_code` | N/A | Scrapfly provides detailed response status in API response | | `custom_google` | N/A | Use standard scraping with `asp=true` for search engines | | `own_proxy` | N/A | Scrapfly uses its own optimized proxy network | ## [Migration Code Examples](#code-examples) Side-by-side code examples showing how to migrate from ScrapingBee to Scrapfly. Select your language below. [ Python ](#python-example) [ JavaScript ](#javascript-example) [ cURL ](#curl-example) ### ScrapingBee ``` from scrapingbee import ScrapingBeeClient client = ScrapingBeeClient( api_key="YOUR_SCRAPINGBEE_API_KEY" ) response = client.get( "https://example.com", params={ "render_js": "true", "stealth_proxy": "true", "premium_proxy": "true", "country_code": "us", "wait": 5000, "wait_for": ".content-loaded" } ) print(response.text) ``` ### Scrapfly ``` from scrapfly import ScrapflyClient, ScrapeConfig client = ScrapflyClient( key="YOUR_SCRAPFLY_API_KEY" ) result = client.scrape(ScrapeConfig( url="https://example.com", render_js=True, asp=True, proxy_pool="public_residential_pool", country="us", rendering_wait=5000, wait_for_selector=".content-loaded" )) print(result.content) ``` ### ScrapingBee ``` const scrapingbee = require('scrapingbee'); const client = new scrapingbee.ScrapingBeeClient( 'YOUR_SCRAPINGBEE_API_KEY' ); const response = await client.get({ url: 'https://example.com', params: { render_js: 'true', stealth_proxy: 'true', premium_proxy: 'true', country_code: 'us', wait: 5000 } }); console.log(response.data); ``` ### Scrapfly ``` const { ScrapflyClient } = require('scrapfly-sdk'); const client = new ScrapflyClient({ key: 'YOUR_SCRAPFLY_API_KEY' }); const result = await client.scrape({ url: 'https://example.com', render_js: true, asp: true, proxy_pool: 'public_residential_pool', country: 'us', rendering_wait: 5000 }); console.log(result.result.content); ``` ### ScrapingBee ``` curl "https://app.scrapingbee.com/api/v1/\ ?api_key=YOUR_API_KEY\ &url=https%3A%2F%2Fexample.com\ &render_js=true\ &stealth_proxy=true\ &premium_proxy=true\ &country_code=us\ &wait=5000" ``` ### Scrapfly ``` curl "https://api.scrapfly.io/scrape\ ?key=YOUR_API_KEY\ &url=https%3A%2F%2Fexample.com\ &render_js=true\ &asp=true\ &proxy_pool=public_residential_pool\ &country=us\ &rendering_wait=5000" ``` ## [🤖 AI Migration Assistant](#ai-migration) Use Claude or ChatGPT to automatically convert your ScrapingBee code to Scrapfly. Copy this prompt and paste it along with your existing code. #### Copy This Prompt ``` I'm migrating from ScrapingBee to Scrapfly. Here's my current code using ScrapingBee's API. Please convert it to use Scrapfly's Python SDK (or JavaScript SDK if my code is in JavaScript). Key parameter mappings: - api_key → key - render_js → render_js (same, but Scrapfly defaults to false) - stealth_proxy=true → asp=True - premium_proxy=true → proxy_pool="public_residential_pool" - country_code → country - wait → rendering_wait - wait_for → wait_for_selector - wait_browser → rendering_stage - session_id → session - forward_headers + Spb-* headers → headers (no prefix needed) - extract_rules → extraction_template - ai_query → extraction_prompt - screenshot=true → screenshots[main]=fullpage - screenshot_selector → screenshots[main]=selector:CSS_SELECTOR - device → os - window_width/height → resolution (e.g., "1920x1080") - json_response=true → (Scrapfly returns structured response by default) - return_page_markdown=true → format="markdown" - return_page_text=true → format="text" Scrapfly SDK Docs (markdown for LLM): https://scrapfly.io/docs/sdk/python?view=markdown Scrapfly API Docs (markdown for LLM): https://scrapfly.io/docs/scrape-api/getting-started?view=markdown My current ScrapingBee code: [PASTE YOUR CODE HERE] ``` **How to Use:**1. Copy the prompt above 2. Open [Claude](https://claude.ai) or [ChatGPT](https://chat.openai.com) 3. Paste the prompt and replace `[PASTE YOUR CODE HERE]` with your ScrapingBee code 4. Review the generated Scrapfly code and test it with your free 1,000 credits **Developer Tools:** Use our [cURL to Python converter](https://scrapfly.io/web-scraping-tools/curl-python) and [selector tester](https://scrapfly.io/web-scraping-tools/css-xpath-tester) to speed up development. ## [Scrapfly Exclusive Features](#exclusive-features) Features available in Scrapfly that aren't available in ScrapingBee. [#### JS Scenarios Automate browser interactions: clicks, form fills, scrolls, and conditional logic. Execute complex workflows without writing browser automation code. ](https://scrapfly.io/products/web-scraping-api) [#### Extraction API AI-powered data extraction with pre-built models for products, articles, jobs, and more. Use LLM prompts for custom extraction without CSS selectors. ](https://scrapfly.io/products/extraction-api) [#### Smart Caching Cache responses to reduce costs and improve response times. Set custom TTL and clear cache on demand. ](https://scrapfly.io/products/web-scraping-api) [#### Crawler API Automated multi-page crawling with intelligent link discovery, sitemap support, and per-URL extraction rules. ](https://scrapfly.io/products/crawler-api) [#### Auto Scroll Automatically scroll pages to trigger lazy-loaded content. Essential for infinite scroll pages like social media feeds. ](https://scrapfly.io/products/web-scraping-api) [#### Webhooks Async processing with delivery guarantees. Get notified when scrapes complete without polling. ](https://scrapfly.io/products/web-scraping-api) ## [Frequently Asked Questions](#faq) #### How do I handle ScrapingBee's `stealth_proxy` + `premium_proxy`? In Scrapfly, `asp=True` handles anti-bot bypass (similar to stealth\_proxy), and `proxy_pool` controls the proxy type: ``` # ScrapingBee stealth_proxy = "true" premium_proxy = "true" # Scrapfly asp = True # Anti-bot bypass proxy_pool = "public_residential_pool" # Residential proxies ``` #### Why is JavaScript rendering different? ScrapingBee defaults to `render_js=true` (5 credits), while Scrapfly defaults to `render_js=false` (1 credit). Explicitly set `render_js=True` in Scrapfly if you need JavaScript rendering. [Learn more about JavaScript rendering ](https://scrapfly.io/docs/scrape-api/javascript-rendering) #### What if I'm using ScrapingBee's CSS extractor? Scrapfly's [Extraction API](https://scrapfly.io/docs/extraction-api/getting-started) is more powerful: - **Template extraction:** Similar to ScrapingBee's CSS extractor - **Auto-extraction:** Automatically detect and extract common data types - **LLM extraction:** Use natural language prompts for complex extraction #### How do I forward custom headers? ScrapingBee requires `forward_headers=true` and `Spb-` prefix. Scrapfly uses the `headers` parameter directly: ``` # ScrapingBee headers={"Spb-Accept-Language": "en-US"} # Scrapfly - simpler! headers={"Accept-Language": "en-US"} ``` #### How do I test my migration? 1. **Sign up for free:** Get 1,000 API credits with no credit card required 2. **Run parallel testing:** Keep ScrapingBee running while testing Scrapfly 3. **Compare results:** Verify that Scrapfly returns the same data 4. **Gradual migration:** Switch traffic gradually (e.g., 10% → 50% → 100%) ## Start Your Migration Today Test Scrapfly on your targets with 1,000 free API credits. No credit card required. - 1,000 free API credits - Full API access - Migration support - Same-day response from our team [ Start For Free ](https://scrapfly.io/register) Need help with migration? [Contact our team ](mailto:support@scrapfly.io)