🚀 We are hiring! See open positions

ScrapingBee to Scrapfly Migration Guide

Complete parameter mapping and code examples for migrating from ScrapingBee to Scrapfly. Most teams complete migration in under 2 hours with zero downtime.

Start For Free - 1,000 Credits Back to ScrapingBee Alternative Page

Complete Parameter Mapping

ScrapingBee and Scrapfly share similar concepts with different parameter names. This table shows exact mappings for all features.

ScrapingBee Parameter Scrapfly Parameter Notes
api_key key API authentication key
url url Target URL to scrape (same)
render_js render_js JavaScript rendering (same name). Note: ScrapingBee defaults to true, Scrapfly defaults to false
stealth_proxy asp Anti-bot bypass (Anti-Scraping Protection). 75 credits → 30+ credits
premium_proxy proxy_pool Use public_residential_pool for residential proxies
country_code country 2-letter ISO country code (e.g., "us", "gb")
wait rendering_wait Wait time in milliseconds before returning response (0-35000 → 1000-25000)
wait_for wait_for_selector CSS selector to wait for before rendering complete
wait_browser rendering_stage Values: domcontentloaded, load, complete
js_scenario js_scenario JavaScript scenario for browser automation (similar JSON format)
session_id session Session name for persistent cookies/state
cookies cookies Custom cookies to send (JSON format)
forward_headers + headers headers Custom HTTP headers (JSON format, no prefix needed)
extract_rules extraction_template Use Scrapfly's Extraction API for structured data
ai_query / ai_extract_rules extraction_prompt AI-powered data extraction with natural language
screenshot screenshots Capture screenshots (use Screenshot API or screenshots param)
screenshot_full_page screenshots[main]=fullpage Capture full page screenshot
screenshot_selector screenshots[main]=selector:CSS_SELECTOR Screenshot a specific element
device os Operating system for browser fingerprint (desktop/mobile)
window_width / window_height resolution Viewport dimensions (e.g., 1920x1080)
json_response format Response format: raw, markdown, text, clean_html, json
return_page_source (default behavior) Scrapfly returns HTML by default
return_page_markdown format=markdown Return page content in Markdown format
return_page_text format=text Return plain text content
block_ads N/A Not directly available; use format_options for content filtering
block_resources N/A Scrapfly optimizes resource loading automatically
timeout timeout Request timeout in milliseconds
transparent_status_code N/A Scrapfly provides detailed response status in API response
custom_google N/A Use standard scraping with asp=true for search engines
own_proxy N/A Scrapfly uses its own optimized proxy network

Migration Code Examples

Side-by-side code examples showing how to migrate from ScrapingBee to Scrapfly. Select your language below.

ScrapingBee
from scrapingbee import ScrapingBeeClient

client = ScrapingBeeClient(
    api_key="YOUR_SCRAPINGBEE_API_KEY"
)

response = client.get(
    "https://example.com",
    params={
        "render_js": "true",
        "stealth_proxy": "true",
        "premium_proxy": "true",
        "country_code": "us",
        "wait": 5000,
        "wait_for": ".content-loaded"
    }
)

print(response.text)
Scrapfly
from scrapfly import ScrapflyClient, ScrapeConfig

client = ScrapflyClient(
    key="YOUR_SCRAPFLY_API_KEY"
)

result = client.scrape(ScrapeConfig(
    url="https://example.com",
    render_js=True,
    asp=True,
    proxy_pool="public_residential_pool",
    country="us",
    rendering_wait=5000,
    wait_for_selector=".content-loaded"
))

print(result.content)
ScrapingBee
const scrapingbee = require('scrapingbee');

const client = new scrapingbee.ScrapingBeeClient(
    'YOUR_SCRAPINGBEE_API_KEY'
);

const response = await client.get({
    url: 'https://example.com',
    params: {
        render_js: 'true',
        stealth_proxy: 'true',
        premium_proxy: 'true',
        country_code: 'us',
        wait: 5000
    }
});

console.log(response.data);
Scrapfly
const { ScrapflyClient } = require('scrapfly-sdk');

const client = new ScrapflyClient({
    key: 'YOUR_SCRAPFLY_API_KEY'
});

const result = await client.scrape({
    url: 'https://example.com',
    render_js: true,
    asp: true,
    proxy_pool: 'public_residential_pool',
    country: 'us',
    rendering_wait: 5000
});

console.log(result.result.content);
ScrapingBee
curl "https://app.scrapingbee.com/api/v1/\
?api_key=YOUR_API_KEY\
&url=https%3A%2F%2Fexample.com\
&render_js=true\
&stealth_proxy=true\
&premium_proxy=true\
&country_code=us\
&wait=5000"
Scrapfly
curl "https://api.scrapfly.io/scrape\
?key=YOUR_API_KEY\
&url=https%3A%2F%2Fexample.com\
&render_js=true\
&asp=true\
&proxy_pool=public_residential_pool\
&country=us\
&rendering_wait=5000"

🤖 AI Migration Assistant

Use Claude or ChatGPT to automatically convert your ScrapingBee code to Scrapfly. Copy this prompt and paste it along with your existing code.

Copy This Prompt

I'm migrating from ScrapingBee to Scrapfly. Here's my current code using ScrapingBee's API.
Please convert it to use Scrapfly's Python SDK (or JavaScript SDK if my code is in JavaScript).

Key parameter mappings:
- api_key → key
- render_js → render_js (same, but Scrapfly defaults to false)
- stealth_proxy=true → asp=True
- premium_proxy=true → proxy_pool="public_residential_pool"
- country_code → country
- wait → rendering_wait
- wait_for → wait_for_selector
- wait_browser → rendering_stage
- session_id → session
- forward_headers + Spb-* headers → headers (no prefix needed)
- extract_rules → extraction_template
- ai_query → extraction_prompt
- screenshot=true → screenshots[main]=fullpage
- screenshot_selector → screenshots[main]=selector:CSS_SELECTOR
- device → os
- window_width/height → resolution (e.g., "1920x1080")
- json_response=true → (Scrapfly returns structured response by default)
- return_page_markdown=true → format="markdown"
- return_page_text=true → format="text"

Scrapfly SDK Docs (markdown for LLM): https://scrapfly.io/docs/sdk/python?view=markdown
Scrapfly API Docs (markdown for LLM): https://scrapfly.io/docs/scrape-api/getting-started?view=markdown

My current ScrapingBee code:
[PASTE YOUR CODE HERE]
How to Use:
  1. Copy the prompt above
  2. Open Claude or ChatGPT
  3. Paste the prompt and replace [PASTE YOUR CODE HERE] with your ScrapingBee code
  4. Review the generated Scrapfly code and test it with your free 1,000 credits

Developer Tools: Use our cURL to Python converter and selector tester to speed up development.

Scrapfly Exclusive Features

Features available in Scrapfly that aren't available in ScrapingBee.

JS Scenarios

Automate browser interactions: clicks, form fills, scrolls, and conditional logic. Execute complex workflows without writing browser automation code.

Extraction API

AI-powered data extraction with pre-built models for products, articles, jobs, and more. Use LLM prompts for custom extraction without CSS selectors.

Smart Caching

Cache responses to reduce costs and improve response times. Set custom TTL and clear cache on demand.

Crawler API

Automated multi-page crawling with intelligent link discovery, sitemap support, and per-URL extraction rules.

Auto Scroll

Automatically scroll pages to trigger lazy-loaded content. Essential for infinite scroll pages like social media feeds.

Webhooks

Async processing with delivery guarantees. Get notified when scrapes complete without polling.

Frequently Asked Questions

How do I handle ScrapingBee's stealth_proxy + premium_proxy?

In Scrapfly, asp=True handles anti-bot bypass (similar to stealth_proxy), and proxy_pool controls the proxy type:

# ScrapingBee
stealth_proxy = "true"
premium_proxy = "true"

# Scrapfly
asp = True                                # Anti-bot bypass
proxy_pool = "public_residential_pool"    # Residential proxies

Why is JavaScript rendering different?

ScrapingBee defaults to render_js=true (5 credits), while Scrapfly defaults to render_js=false (1 credit). Explicitly set render_js=True in Scrapfly if you need JavaScript rendering.

Learn more about JavaScript rendering

What if I'm using ScrapingBee's CSS extractor?

Scrapfly's Extraction API is more powerful:

  • Template extraction: Similar to ScrapingBee's CSS extractor
  • Auto-extraction: Automatically detect and extract common data types
  • LLM extraction: Use natural language prompts for complex extraction

How do I forward custom headers?

ScrapingBee requires forward_headers=true and Spb- prefix. Scrapfly uses the headers parameter directly:

# ScrapingBee
headers={"Spb-Accept-Language": "en-US"}

# Scrapfly - simpler!
headers={"Accept-Language": "en-US"}

How do I test my migration?
  1. Sign up for free: Get 1,000 API credits with no credit card required
  2. Run parallel testing: Keep ScrapingBee running while testing Scrapfly
  3. Compare results: Verify that Scrapfly returns the same data
  4. Gradual migration: Switch traffic gradually (e.g., 10% → 50% → 100%)

Start Your Migration Today

Test Scrapfly on your targets with 1,000 free API credits. No credit card required.

  • 1,000 free API credits
  • Full API access
  • Migration support
  • Same-day response from our team
Start For Free

Need help with migration? Contact our team