# Scrapfly Documentation

## Table of Contents

### Dashboard

- [Intro](https://scrapfly.io/docs)
- [Project](https://scrapfly.io/docs/project)
- [Account](https://scrapfly.io/docs/account)
- [Workspace & Team](https://scrapfly.io/docs/workspace-and-team)
- [Billing](https://scrapfly.io/docs/billing)

### Products

#### MCP Server

- [Getting Started](https://scrapfly.io/docs/mcp/getting-started)
- [Tools & API Spec](https://scrapfly.io/docs/mcp/tools)
- [Authentication](https://scrapfly.io/docs/mcp/authentication)
- [Examples & Use Cases](https://scrapfly.io/docs/mcp/examples)
- [FAQ](https://scrapfly.io/docs/mcp/faq)
##### Integrations

- [Overview](https://scrapfly.io/docs/mcp/integrations)
- [Claude Desktop](https://scrapfly.io/docs/mcp/integrations/claude-desktop)
- [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code)
- [ChatGPT](https://scrapfly.io/docs/mcp/integrations/chatgpt)
- [Cursor](https://scrapfly.io/docs/mcp/integrations/cursor)
- [Cline](https://scrapfly.io/docs/mcp/integrations/cline)
- [Windsurf](https://scrapfly.io/docs/mcp/integrations/windsurf)
- [Zed](https://scrapfly.io/docs/mcp/integrations/zed)
- [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code)
- [VS Code](https://scrapfly.io/docs/mcp/integrations/vscode)
- [LangChain](https://scrapfly.io/docs/mcp/integrations/langchain)
- [LlamaIndex](https://scrapfly.io/docs/mcp/integrations/llamaindex)
- [CrewAI](https://scrapfly.io/docs/mcp/integrations/crewai)
- [OpenAI](https://scrapfly.io/docs/mcp/integrations/openai)
- [n8n](https://scrapfly.io/docs/mcp/integrations/n8n)
- [Make](https://scrapfly.io/docs/mcp/integrations/make)
- [Zapier](https://scrapfly.io/docs/mcp/integrations/zapier)
- [Vapi AI](https://scrapfly.io/docs/mcp/integrations/vapi)
- [Agent Builder](https://scrapfly.io/docs/mcp/integrations/agent-builder)
- [Custom Client](https://scrapfly.io/docs/mcp/integrations/custom-client)


#### Web Scraping API

- [Getting Started](https://scrapfly.io/docs/scrape-api/getting-started)
- [API Specification]()
- [Monitoring](https://scrapfly.io/docs/monitoring)
- [Customize Request](https://scrapfly.io/docs/scrape-api/custom)
- [Debug](https://scrapfly.io/docs/scrape-api/debug)
- [Anti Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection)
- [Proxy](https://scrapfly.io/docs/scrape-api/proxy)
- [Proxy Mode](https://scrapfly.io/docs/scrape-api/proxy-mode)
- [Proxy Mode - Screaming Frog](https://scrapfly.io/docs/scrape-api/proxy-mode/screaming-frog)
- [Proxy Mode - Apify](https://scrapfly.io/docs/scrape-api/proxy-mode/apify)
- [(Auto) Data Extraction](https://scrapfly.io/docs/scrape-api/extraction)
- [Javascript Rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering)
- [Javascript Scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario)
- [SSL](https://scrapfly.io/docs/scrape-api/ssl)
- [DNS](https://scrapfly.io/docs/scrape-api/dns)
- [Cache](https://scrapfly.io/docs/scrape-api/cache)
- [Batch (Multi-URL Scraping)](https://scrapfly.io/docs/scrape-api/batch)
- [Session](https://scrapfly.io/docs/scrape-api/session)
- [Webhook](https://scrapfly.io/docs/scrape-api/webhook)
- [Schedule](https://scrapfly.io/docs/scrape-api/schedule)
- [Screenshot](https://scrapfly.io/docs/scrape-api/screenshot)
- [Errors](https://scrapfly.io/docs/scrape-api/errors)
- [Timeout](https://scrapfly.io/docs/scrape-api/understand-timeout)
- [Throttling](https://scrapfly.io/docs/throttling)
- [Troubleshoot](https://scrapfly.io/docs/scrape-api/troubleshoot)
- [Billing](https://scrapfly.io/docs/scrape-api/billing)
- [FAQ](https://scrapfly.io/docs/scrape-api/faq)

#### Crawler API

- [Getting Started](https://scrapfly.io/docs/crawler-api/getting-started)
- [API Specification]()
- [Retrieving Results](https://scrapfly.io/docs/crawler-api/results)
- [WARC Format](https://scrapfly.io/docs/crawler-api/warc-format)
- [Data Extraction](https://scrapfly.io/docs/crawler-api/extraction-rules)
- [Webhook](https://scrapfly.io/docs/crawler-api/webhook)
- [Schedule](https://scrapfly.io/docs/crawler-api/schedule)
- [Billing](https://scrapfly.io/docs/crawler-api/billing)
- [Errors](https://scrapfly.io/docs/crawler-api/errors)
- [Troubleshoot](https://scrapfly.io/docs/crawler-api/troubleshoot)
- [FAQ](https://scrapfly.io/docs/crawler-api/faq)

#### Screenshot API

- [Getting Started](https://scrapfly.io/docs/screenshot-api/getting-started)
- [API Specification]()
- [Accessibility Testing](https://scrapfly.io/docs/screenshot-api/accessibility)
- [Webhook](https://scrapfly.io/docs/screenshot-api/webhook)
- [Schedule](https://scrapfly.io/docs/screenshot-api/schedule)
- [Billing](https://scrapfly.io/docs/screenshot-api/billing)
- [Errors](https://scrapfly.io/docs/screenshot-api/errors)

#### Extraction API

- [Getting Started](https://scrapfly.io/docs/extraction-api/getting-started)
- [API Specification]()
- [Rules Template](https://scrapfly.io/docs/extraction-api/rules-and-template)
- [LLM Extraction](https://scrapfly.io/docs/extraction-api/llm-prompt)
- [AI Auto Extraction](https://scrapfly.io/docs/extraction-api/automatic-ai)
- [Webhook](https://scrapfly.io/docs/extraction-api/webhook)
- [Billing](https://scrapfly.io/docs/extraction-api/billing)
- [Errors](https://scrapfly.io/docs/extraction-api/errors)
- [FAQ](https://scrapfly.io/docs/extraction-api/faq)

#### Proxy Saver

- [Getting Started](https://scrapfly.io/docs/proxy-saver/getting-started)
- [Fingerprints](https://scrapfly.io/docs/proxy-saver/fingerprints)
- [Optimizations](https://scrapfly.io/docs/proxy-saver/optimizations)
- [SSL Certificates](https://scrapfly.io/docs/proxy-saver/certificates)
- [Protocols](https://scrapfly.io/docs/proxy-saver/protocols)
- [Pacfile](https://scrapfly.io/docs/proxy-saver/pacfile)
- [Secure Credentials](https://scrapfly.io/docs/proxy-saver/security)
- [Billing](https://scrapfly.io/docs/proxy-saver/billing)

#### Cloud Browser API

- [Getting Started](https://scrapfly.io/docs/cloud-browser-api/getting-started)
- [Proxy & Geo-Targeting](https://scrapfly.io/docs/cloud-browser-api/proxy)
- [Unblock API](https://scrapfly.io/docs/cloud-browser-api/unblock)
- [Captcha Solver](https://scrapfly.io/docs/cloud-browser-api/captcha-solver)
- [File Downloads](https://scrapfly.io/docs/cloud-browser-api/file-downloads)
- [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume)
- [Human-in-the-Loop](https://scrapfly.io/docs/cloud-browser-api/human-in-the-loop)
- [Debug Mode](https://scrapfly.io/docs/cloud-browser-api/debug-mode)
- [Bring Your Own Proxy](https://scrapfly.io/docs/cloud-browser-api/bring-your-own-proxy)
- [Browser Extensions](https://scrapfly.io/docs/cloud-browser-api/extensions)
- [Native Browser MCP](https://scrapfly.io/docs/cloud-browser-api/mcp)
- [DevTools Protocol](https://scrapfly.io/docs/cloud-browser-api/cdp-reference)
##### Integrations

- [Puppeteer](https://scrapfly.io/docs/cloud-browser-api/puppeteer)
- [Playwright](https://scrapfly.io/docs/cloud-browser-api/playwright)
- [Selenium](https://scrapfly.io/docs/cloud-browser-api/selenium)
- [Vercel Agent Browser](https://scrapfly.io/docs/cloud-browser-api/agent-browser)
- [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use)
- [Stagehand](https://scrapfly.io/docs/cloud-browser-api/stagehand)

- [Billing](https://scrapfly.io/docs/cloud-browser-api/billing)
- [Errors](https://scrapfly.io/docs/cloud-browser-api/errors)


### Tools

- [Antibot Detector](https://scrapfly.io/docs/tools/antibot-detector)

### SDK

- [Golang](https://scrapfly.io/docs/sdk/golang)
- [Python](https://scrapfly.io/docs/sdk/python)
- [Rust](https://scrapfly.io/docs/sdk/rust)
- [TypeScript](https://scrapfly.io/docs/sdk/typescript)
- [Scrapy](https://scrapfly.io/docs/sdk/scrapy)

### Integrations

- [Getting Started](https://scrapfly.io/docs/integration/getting-started)
- [LangChain](https://scrapfly.io/docs/integration/langchain)
- [LlamaIndex](https://scrapfly.io/docs/integration/llamaindex)
- [CrewAI](https://scrapfly.io/docs/integration/crewai)
- [Zapier](https://scrapfly.io/docs/integration/zapier)
- [Make](https://scrapfly.io/docs/integration/make)
- [n8n](https://scrapfly.io/docs/integration/n8n)

### Academy

- [Overview](https://scrapfly.io/academy)
- [Web Scraping Overview](https://scrapfly.io/academy/scraping-overview)
- [Tools](https://scrapfly.io/academy/tools-overview)
- [Reverse Engineering](https://scrapfly.io/academy/reverse-engineering)
- [Static Scraping](https://scrapfly.io/academy/static-scraping)
- [HTML Parsing](https://scrapfly.io/academy/html-parsing)
- [Dynamic Scraping](https://scrapfly.io/academy/dynamic-scraping)
- [Hidden API Scraping](https://scrapfly.io/academy/hidden-api-scraping)
- [Headless Browsers](https://scrapfly.io/academy/headless-browsers)
- [Hidden Web Data](https://scrapfly.io/academy/hidden-web-data)
- [JSON Parsing](https://scrapfly.io/academy/json-parsing)
- [Data Processing](https://scrapfly.io/academy/data-processing)
- [Scaling](https://scrapfly.io/academy/scaling)
- [Walkthrough Summary](https://scrapfly.io/academy/walkthrough-summary)
- [Scraper Blocking](https://scrapfly.io/academy/scraper-blocking)
- [Proxies](https://scrapfly.io/academy/proxies)

---

#  Proxy Saver Protocols 

 The Proxy Saver supports all standard and modern proxy protocols, ensuring compatibility with any browser or web scraping tool. Below are the details and specificities of each protocol's usage.

> While all the provided examples account for the possibility that the Proxy Saver certificates may not be properly installed, the recommended approach is to [configure them correctly](https://scrapfly.io/docs/proxy-saver/certificates) rather than bypassing SSL errors.

## HTTP

 HTTP is the simplest proxy protocol and is universally compatible with all browsers and libraries.
 Below are some examples:

- [Curl](#basic-use-curl-http)
- [Playwright (js)](#scrape-http-proxy-playwright-js)
- [Playwright (python)](#scrape-http-proxy-playwright-python)
 
 

 ```
curl \
--insecure \
--proxy http://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything

```

 

   

 

 

 ```
const {chromium} = require("playwright");

(async () => {
    try {
        const browser = await chromium.launch({
            proxy: {
                server: 'http://proxy-saver.scrapfly.io:3333',
                username: 'proxyId-XXX',
                password: 'scp-live-XXX'
            }
        });
        const context = await browser.newContext({ignoreHTTPSErrors: true});
        const page = await context.newPage();

        await page.goto("https://httpbin.dev/anything");

        const html = await page.content();
        console.log(html);

        await browser.close();
    } catch (error) {
        console.error("An error occurred:", error);
    }
})();

```

 

   

 

 

 ```
from playwright.sync_api import sync_playwright

def main():
    try:
        with sync_playwright() as p:
            browser = p.chromium.launch(
                proxy={
                    "server": "http://proxy-saver.scrapfly.io:3333",
                    "username": "proxyId-XXX",
                    "password": "scp-live-XXX",
                }
            )

        context = browser.new_context(ignore_https_errors=True)
        page = context.new_page()

        page.goto("https://httpbin.dev/anything")

        html = page.content()
        print(html)
    except Exception as e:
        print("An error occurred:", e)
    finally:
        browser.close()

if __name__ == "__main__":
    main()

```

 

   

 

 

 

 

## HTTPS / HTTP2

 Using HTTP as a proxy protocol is straightforward, but opting for HTTPS ensures that your credentials are securely encrypted from the very start of the CONNECT phase.

 By default, curl will use HTTP/1.1 over TLS unless you have a recent version that supports HTTP/2 and explicitly specify the necessary flags. Chrome, on the other hand, defaults to HTTP/2 but will gracefully downgrade to HTTP/1.1 over TLS if needed.

 The Proxy Saver natively supports HTTP/2 while also maintaining compatibility with HTTP/1.1 over TLS to accommodate the broadest range of browser implementations.

- [Curl (HTTPS)](#basic-use-curl-https)
- [Curl (HTTP2)](#basic-use-curl-http2)
- [Playwright (js)](#scrape-https-proxy-playwright-js)
- [Playwright (python)](#scrape-https-proxy-playwright-python)
 
 

 ```
curl \
--insecure --proxy-insecure \
--proxy https://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything

```

 

   

 

 

 ```
curl \
--insecure --proxy-insecure \
--proxy-http2 \
--proxy https://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything

```

 

   

 

 

 ```
const {chromium} = require("playwright");

(async () => {
    try {
        const browser = await chromium.launch({
            proxy: {
                server: 'https://proxy-saver.scrapfly.io:3333',
                username: 'proxyId-XXX',
                password: 'scp-live-XXX'
            },
            args: ['--ignore-certificate-errors'],
        });

        const context = await browser.newContext({ignoreHTTPSErrors: true});
        const page = await context.newPage();

        await page.goto("https://httpbin.dev/anything");

        const html = await page.content();
        console.log(html);

        await browser.close();
    } catch (error) {
        console.error("An error occurred:", error);
    }
})();

```

 

   

 

 

 ```
from playwright.sync_api import sync_playwright

def main():
    try:
        with sync_playwright() as p:
            browser = p.chromium.launch(
                proxy={
                    "server": "https://proxy-saver.scrapfly.io:3333",
                    "username": "proxyId-XXX",
                    "password": "scp-live-XXX",
                },
                args=["--ignore-certificate-errors"]
            )

        context = browser.new_context(ignore_https_errors=True)
        page = context.new_page()

        page.goto("https://httpbin.dev/anything")

        html = page.content()
        print(html)
    except Exception as e:
        print("An error occurred:", e)
    finally:
        browser.close()

if __name__ == "__main__":
    main()



```

 

   

 

 

 

 

## SOCKS5

 Socks5/h is another popular protocol used for HTTP proxying

### Socks5 auth mechanism

 For browsers and libraries that support native SOCKS5 authentication, the Proxy Saver does as well, ensuring seamless integration.

- [Curl (SOCKS5 Auth)](#basic-use-curl-https)
 
 

 ```
curl \
--insecure \
--proxy socks5h://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything

```

 

   

 

 

 

 

 However, not all browsers support this mechanism.

> While Chrome and Chromium do not support the SOCKS5 authentication mechanism (and have no plans to), Scrapfly offers a reliable workaround.

 For browsers and libraries that do not support SOCKS5 authentication natively, Scrapfly offers a simple workaround using a custom header. 
 Simply include your authentication string in the `X-SAVER-AUTH` header. 
 This custom authentication mechanism enables the use of the SOCKS5 protocol with any library that supports it.

 Below are some usage examples:

- [Curl (X-SAVER-AUTH)](#basic-use-curl-socks5h-x-auth)
- [Playwright (js)](#scrape-socks5-proxy-playwright-js)
- [Playwright (python)](#scrape-socks5-proxy-playwright-python)
 
 

 ```
curl \
--insecure \
--proxy socks5h://proxy-saver.scrapfly.io:3333 \
--header "X-SAVER-AUTH: proxyId-XXX:scp-live-XXX" \
https://httpbin.dev/anything

```

 

   

 

 

 ```
const { chromium } = require("playwright");

const xSaverAuth = { 'X-SAVER-AUTH': 'proxyId-XXX:scp-live-XXX'};

(async () => {
    try {
        const browser = await chromium.launch({
            proxy: {
                server: 'socks5://proxy-saver.scrapfly.io:3333',
            }
        });
        const context = await browser.newContext({ ignoreHTTPSErrors: true });
        await context.setExtraHTTPHeaders(xSaverAuth);
        const page = await context.newPage();

        await page.goto("https://httpbin.dev/anything");

        const html = await page.content();
        console.log(html);

        await browser.close();
    } catch (error) {
        console.error("An error occurred:", error);
    }
})();

```

 

   

 

 

 ```
from playwright.sync_api import sync_playwright

def main():
    try:
        with sync_playwright() as p:
            browser = p.chromium.launch(
                proxy={
                    "server": "socks5://proxy-saver.scrapfly.io:3333",
                }
            )

        context = browser.new_context(ignore_https_errors=True)
        context.set_extra_http_headers({
            "X-SAVER-AUTH": "proxyId-XXX:scp-live-XXX"
        })

        page = context.new_page()
        page.goto("https://httpbin.dev/anything")

        html = page.content()
        print(html)
    except Exception as e:
        print("An error occurred:", e)
    finally:
        browser.close()

if __name__ == "__main__":
    main()



```