Web Scraping API

For Developers

Get the best success rate with our battle-tested solution

  • Automatic Proxy Rotation
  • Bypass anti bot solutions
  • Managed Headless Browsers

You can handle any increase in workload without worrying about infrastructure.

Try for free Capterra review badge
99.99%
Uptime
1PB+
Data Transferred
5B+
Success Requests
Monthly
curl -i "https://api.scrapfly.io/scrape?key=APIKEY&url=https://httpbin.dev/anything"
HTTP/2 200
content-type: application/json; charset=utf-8
x-scrapfly-account-concurrent-usage: 1
x-scrapfly-account-remaining-concurrent-usage: 4
x-scrapfly-api-cost: 1
x-scrapfly-remaining-api-credit: 9982276
x-scrapfly-response-time: 1.090000
{
    "config": {...},
    "result": {
        "content": "{\n  \"args\": {}, \n  \"data\": \"\", \n...
        "size": 1094,
        "ssl": null,
        "status": "DONE",
        "status_code": 200,
        "success": true,
        "url": "https://httpbin.dev/anything"
        "format": "text",
        "iframes": [],
        "log_url": "https://scrapfly.io/dashboard/monitoring/log/07f459cd-5b8c-4c5a-9649-c6bb70c1314e",
        "request_headers" : { ... },
        "response_headers" : { ... },
        "screenshots" : [],
        ...
    },
    ...
}}

Features

Anti Scraping Protection

Anti-Scraping Protection

&asp=true

Our ASP bypasses major anti-scraping protection by shielding your scraping activities, and we consistently update it to ensure its effectiveness.

Screenshot

Screenshot

&screenshots[test]=fullpage

Take one or multiple screenshots of page, full page or scoped on element

Ban Detection

Ban Detection

IP failover / cooling - Intelligent distribution backed by a statistical model

Real Time Dashboard

Real-Time Dashboard

Monitor, analyze, introspect, debug, screenshot, replay from dedicated URL to fix

Automatic IP Roatation

Proxies

&proxy_pool=public_residential_pool

Datacenter, Residential proxies pool available. Automatic proxy network type upgrade to bypass with anti bot

Presistent Sessions

Persistent Sessions

&session=my-session

Persist session on demand and use the same context across requests

Javascript Rendering

Javascript Rendering

&render_js=true

Scrape with a real browser, execute your own javascript

Caching System

Caching System

&cache=true

On-demand, rebuild dataset from scratch faster without overloading an upstream website

Async

Webhook

&webhook=my-endpoint-name

Send scrape order and receive result through webhook notification when scrape is finished with intelligent backoff, just in case, you know

Fully Customizable

Fully Customizable

&headers[X-Custom]=test

Customs headers, post rendering delay, HTTP method (GET, POST, PUT, PATCH, HEAD)

Simple & Predictable Billing

How many scrape per month do you need?

Get started with 1,000 free Scrape API Credits. Try Scrapfly for free
Custom Subscription
∞ /mo
Custom
ENTERPRISE Subscription
$500/mo
Enterprise
STARTUP Subscription
$250/mo
Startup
PRO Subscription
$100/mo
Pro
DISCOVERY Subscription
$30/mo
Discovery
Included API Credits ∞ 5,500,000 2,500,000 1,000,000 200,000
Extra API Credits ∞ per 10k $1.20 per 10k $2.00 per 10k $3.50 per 10k ✖
Screenshot API ∞ 91,667 41,667 16,667 3,333
Concurrent Request ∞ 100 50 20 5
Log Retention ∞ weeks 4 weeks 3 weeks 2 weeks 1 weeks
Anti Scraping Protection ✓ ✓ ✓ ✓ ✓
Residential Proxy ✓ ✓ ✓ ✓ ✓
Geo targeting ✓ ✓ ✓ ✓ ✓
Javascript Rendering ✓ ✓ ✓ ✓ ✓
Support Premium Support Premium Support Standard Support Standard Support Basic Support
Try with 1,000 Free API Credits

* Price may vary with tax

Why Working With Us?

Scrapfly was originally an insider product and battle tested before becoming publicly accessible in early 2017. It was then opened to everyone in 2020.

Our years of experience in the industry and use of the service for our own clients has built trust and confidence in our product and the value we deliver.

Our mission is to simplify the work of data engineers and analysts to access the data. The internet is the largest database in the world, yet it remains the least accessible, both technically and legally.

Why our customers appreciate us

  1. The success rate.
  2. The reliability.
  3. The speed at which we can update and adapt our solution is a recurring factor in why our customers choose our service.
  4. The technical ability of our support.
  5. Our documentation is driven by playable examples.

Key Benefits

  1. Reduced retry attempts result in lower consumption of compute resources, bandwidth and human resources.
  2. Developers no longer have to spend time on recurring updates and research on why they are blocked.
  3. Faster results - near real time results ability.
  4. Built in monitoring and sharable config / logs to communicate.
  5. Eliminates the need to hire top-tier data engineers or security researchers to extract data from difficult websites or update anti-bot bypass

Bypass advanced website anti-bot systems

  • Proprietary Solution
  • Dynamic Fingerprint
  • Bypass Anti Bots
  • Accurate and reliable results

Auto Updated

  • 5 new version of our scrape engine is deployed on average each day
  • 3 Business Days, that the average time to overcome anti-bot update
  • 100M+ Browser and device fingerprint are derived from real user fingerprint each day
  • 30M+ of proxies IPs are rotated each hours on our public pool

Maintaining confidentiality

  • Your Name is not used in any marketing communication
  • No result caching
  • No data resell
  • Sensitive parameters are scrubbed

Proxy Quality

  • Proxy origin verified
  • Quality monitored
  • Proxy pools are ethically selected and secure

Strict vetting process

  • Client Verification
  • Prevent and monitor illegal activites

Technical support

  • We do not externalize our support
  • No tiering level to pass before getting a working solution
  • Each support agent are technically trained and have a data extraction background