Scrapfly Developer Hub

Scrape any page, bypass blocking, control a real web browser and get output results html, markdown or json:

from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse

scrapfly = ScrapflyClient(key='')

api_response: ScrapeApiResponse = scrapfly.scrape(
    ScrapeConfig(
        url='https://news.ycombinator.com/'
        # enable cloud browsers
        render_js=True,
        # bypass anti-scraping protection
        asp=True,
        # much more!
    )
)

print(api_response.selector('#title'))

Screenshot capture made easy - capture any area, full page, scroll and more:

Intro & API Spec

from scrapfly import ScrapflyClient, ScreenshotConfig

scrapfly = ScrapflyClient(key='')

api_response = scrapfly.screenshot(
    ScreenshotConfig(
        url='https://news.ycombinator.com/',
        # automatically scroll
        auto_scroll=True,
        # capture specific areas
        capture=".review",
))

with open("screenshot.png", "wb") as f:
    f.write(api_response.image)

Extract data from documents using AI, LLMs and extraction templates:

Intro & API Spec

from scrapfly import ExtractionConfig, ScrapflyClient

scrapfly = ScrapflyClient(key='')

api_response = scrapfly.scrape(
    ExtractionConfig(
        content_type="text/html",
            body="""
                <html>
                <body><h1>Hello, World!</h1></body>
                </html>
            """,
            # ask LLMs:
            extraction_prompt="What is the title of the page?",
            # or find objects automatically:
            extraction_model="product",
        )
    )
print(api_response)

Power up your proxies with bandwidth, speed and stability improvements.

Intro & API Spec

curl \
--insecure \
--proxy http://proxyId-XXX-FpImpersonate-chrome116:API_KEY@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything

Scrapfly APIs for every step of your data pipeline - start with step-by-step onboarding!

Start with Python Start with Typescript

Integration with Popular Tools

Integrate into your automated workflows
Try our no-code integrations with popular no-code platforms like Zapier, Make.com, n8n and others.

Already have a tech stack?
See our existing integrations with popular libraries like LangChain, LamaIndex, Scrapy and more!

Integrate our SDKs

Python SDK

Typescript SDK

Scrapy

Explore using Interactive Examples

Discover intuitive and interactive examples. We have developed a playground to test our solution without writing a single line and generate code in many languages to illustrate.

Discover the Interactive Player*
*Requires an account

Web Scraping Academy NEW

Learn everything about web scraping through Scrapfly Academy - a free educational platform for diving into web scraping.

Start Learning

Speedup with Web Scraping Tools

Our internal scraping tools are open for everyone! Use them to test your scrapers, understand bot detection or just learn everything web scraping.

Discover Web Scraping Tools