# Scrapfly Documentation ## Table of Contents ### Dashboard - [Intro](https://scrapfly.io/docs) - [Project](https://scrapfly.io/docs/project) - [Account](https://scrapfly.io/docs/account) - [Workspace & Team](https://scrapfly.io/docs/workspace-and-team) - [Billing](https://scrapfly.io/docs/billing) ### Products #### MCP Server - [Getting Started](https://scrapfly.io/docs/mcp/getting-started) - [Tools & API Spec](https://scrapfly.io/docs/mcp/tools) - [Authentication](https://scrapfly.io/docs/mcp/authentication) - [Examples & Use Cases](https://scrapfly.io/docs/mcp/examples) - [FAQ](https://scrapfly.io/docs/mcp/faq) ##### Integrations - [Overview](https://scrapfly.io/docs/mcp/integrations) - [Claude Desktop](https://scrapfly.io/docs/mcp/integrations/claude-desktop) - [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code) - [ChatGPT](https://scrapfly.io/docs/mcp/integrations/chatgpt) - [Cursor](https://scrapfly.io/docs/mcp/integrations/cursor) - [Cline](https://scrapfly.io/docs/mcp/integrations/cline) - [Windsurf](https://scrapfly.io/docs/mcp/integrations/windsurf) - [Zed](https://scrapfly.io/docs/mcp/integrations/zed) - [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code) - [VS Code](https://scrapfly.io/docs/mcp/integrations/vscode) - [LangChain](https://scrapfly.io/docs/mcp/integrations/langchain) - [LlamaIndex](https://scrapfly.io/docs/mcp/integrations/llamaindex) - [CrewAI](https://scrapfly.io/docs/mcp/integrations/crewai) - [OpenAI](https://scrapfly.io/docs/mcp/integrations/openai) - [n8n](https://scrapfly.io/docs/mcp/integrations/n8n) - [Make](https://scrapfly.io/docs/mcp/integrations/make) - [Zapier](https://scrapfly.io/docs/mcp/integrations/zapier) - [Vapi AI](https://scrapfly.io/docs/mcp/integrations/vapi) - [Agent Builder](https://scrapfly.io/docs/mcp/integrations/agent-builder) - [Custom Client](https://scrapfly.io/docs/mcp/integrations/custom-client) #### Web Scraping API - [Getting Started](https://scrapfly.io/docs/scrape-api/getting-started) - [API Specification](https://scrapfly.io/docs/scrape-api/specification) - [Monitoring](https://scrapfly.io/docs/monitoring) - [Customize Request](https://scrapfly.io/docs/scrape-api/custom) - [Debug](https://scrapfly.io/docs/scrape-api/debug) - [Anti Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - [Proxy](https://scrapfly.io/docs/scrape-api/proxy) - [Proxy Mode](https://scrapfly.io/docs/scrape-api/proxy-mode) - [Proxy Mode - Screaming Frog](https://scrapfly.io/docs/scrape-api/proxy-mode/screaming-frog) - [Proxy Mode - Apify](https://scrapfly.io/docs/scrape-api/proxy-mode/apify) - [(Auto) Data Extraction](https://scrapfly.io/docs/scrape-api/extraction) - [Javascript Rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - [Javascript Scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario) - [SSL](https://scrapfly.io/docs/scrape-api/ssl) - [DNS](https://scrapfly.io/docs/scrape-api/dns) - [Cache](https://scrapfly.io/docs/scrape-api/cache) - [Session](https://scrapfly.io/docs/scrape-api/session) - [Webhook](https://scrapfly.io/docs/scrape-api/webhook) - [Screenshot](https://scrapfly.io/docs/scrape-api/screenshot) - [Errors](https://scrapfly.io/docs/scrape-api/errors) - [Timeout](https://scrapfly.io/docs/scrape-api/understand-timeout) - [Throttling](https://scrapfly.io/docs/throttling) - [Troubleshoot](https://scrapfly.io/docs/scrape-api/troubleshoot) - [Billing](https://scrapfly.io/docs/scrape-api/billing) - [FAQ](https://scrapfly.io/docs/scrape-api/faq) #### Crawler API - [Getting Started](https://scrapfly.io/docs/crawler-api/getting-started) - [API Specification](https://scrapfly.io/docs/crawler-api/specification) - [Retrieving Results](https://scrapfly.io/docs/crawler-api/results) - [WARC Format](https://scrapfly.io/docs/crawler-api/warc-format) - [Data Extraction](https://scrapfly.io/docs/crawler-api/extraction-rules) - [Webhook](https://scrapfly.io/docs/crawler-api/webhook) - [Billing](https://scrapfly.io/docs/crawler-api/billing) - [Errors](https://scrapfly.io/docs/crawler-api/errors) - [Troubleshoot](https://scrapfly.io/docs/crawler-api/troubleshoot) - [FAQ](https://scrapfly.io/docs/crawler-api/faq) #### Screenshot API - [Getting Started](https://scrapfly.io/docs/screenshot-api/getting-started) - [API Specification](https://scrapfly.io/docs/screenshot-api/specification) - [Accessibility Testing](https://scrapfly.io/docs/screenshot-api/accessibility) - [Webhook](https://scrapfly.io/docs/screenshot-api/webhook) - [Billing](https://scrapfly.io/docs/screenshot-api/billing) - [Errors](https://scrapfly.io/docs/screenshot-api/errors) #### Extraction API - [Getting Started](https://scrapfly.io/docs/extraction-api/getting-started) - [API Specification](https://scrapfly.io/docs/extraction-api/specification) - [Rules Template](https://scrapfly.io/docs/extraction-api/rules-and-template) - [LLM Extraction](https://scrapfly.io/docs/extraction-api/llm-prompt) - [AI Auto Extraction](https://scrapfly.io/docs/extraction-api/automatic-ai) - [Webhook](https://scrapfly.io/docs/extraction-api/webhook) - [Billing](https://scrapfly.io/docs/extraction-api/billing) - [Errors](https://scrapfly.io/docs/extraction-api/errors) - [FAQ](https://scrapfly.io/docs/extraction-api/faq) #### Proxy Saver - [Getting Started](https://scrapfly.io/docs/proxy-saver/getting-started) - [Fingerprints](https://scrapfly.io/docs/proxy-saver/fingerprints) - [Optimizations](https://scrapfly.io/docs/proxy-saver/optimizations) - [SSL Certificates](https://scrapfly.io/docs/proxy-saver/certificates) - [Protocols](https://scrapfly.io/docs/proxy-saver/protocols) - [Pacfile](https://scrapfly.io/docs/proxy-saver/pacfile) - [Secure Credentials](https://scrapfly.io/docs/proxy-saver/security) - [Billing](https://scrapfly.io/docs/proxy-saver/billing) #### Cloud Browser API - [Getting Started](https://scrapfly.io/docs/cloud-browser-api/getting-started) - [Proxy & Geo-Targeting](https://scrapfly.io/docs/cloud-browser-api/proxy) - [Unblock API](https://scrapfly.io/docs/cloud-browser-api/unblock) - [File Downloads](https://scrapfly.io/docs/cloud-browser-api/file-downloads) - [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume) - [Human-in-the-Loop](https://scrapfly.io/docs/cloud-browser-api/human-in-the-loop) - [Debug Mode](https://scrapfly.io/docs/cloud-browser-api/debug-mode) - [Bring Your Own Proxy](https://scrapfly.io/docs/cloud-browser-api/bring-your-own-proxy) - [Browser Extensions](https://scrapfly.io/docs/cloud-browser-api/extensions) ##### Integrations - [Puppeteer](https://scrapfly.io/docs/cloud-browser-api/puppeteer) - [Playwright](https://scrapfly.io/docs/cloud-browser-api/playwright) - [Selenium](https://scrapfly.io/docs/cloud-browser-api/selenium) - [Vercel Agent Browser](https://scrapfly.io/docs/cloud-browser-api/agent-browser) - [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use) - [Stagehand](https://scrapfly.io/docs/cloud-browser-api/stagehand) - [Vibium](https://scrapfly.io/docs/cloud-browser-api/vibium) - [Billing](https://scrapfly.io/docs/cloud-browser-api/billing) - [Errors](https://scrapfly.io/docs/cloud-browser-api/errors) ### Tools - [Antibot Detector](https://scrapfly.io/docs/tools/antibot-detector) ### SDK - [Golang](https://scrapfly.io/docs/sdk/golang) - [Python](https://scrapfly.io/docs/sdk/python) - [TypeScript](https://scrapfly.io/docs/sdk/typescript) - [Scrapy](https://scrapfly.io/docs/sdk/scrapy) ### Integrations - [Getting Started](https://scrapfly.io/docs/integration/getting-started) - [LangChain](https://scrapfly.io/docs/integration/langchain) - [LlamaIndex](https://scrapfly.io/docs/integration/llamaindex) - [CrewAI](https://scrapfly.io/docs/integration/crewai) - [Zapier](https://scrapfly.io/docs/integration/zapier) - [Make](https://scrapfly.io/docs/integration/make) - [n8n](https://scrapfly.io/docs/integration/n8n) ### Academy - [Overview](https://scrapfly.io/academy) - [Web Scraping Overview](https://scrapfly.io/academy/scraping-overview) - [Tools](https://scrapfly.io/academy/tools-overview) - [Reverse Engineering](https://scrapfly.io/academy/reverse-engineering) - [Static Scraping](https://scrapfly.io/academy/static-scraping) - [HTML Parsing](https://scrapfly.io/academy/html-parsing) - [Dynamic Scraping](https://scrapfly.io/academy/dynamic-scraping) - [Hidden API Scraping](https://scrapfly.io/academy/hidden-api-scraping) - [Headless Browsers](https://scrapfly.io/academy/headless-browsers) - [Hidden Web Data](https://scrapfly.io/academy/hidden-web-data) - [JSON Parsing](https://scrapfly.io/academy/json-parsing) - [Data Processing](https://scrapfly.io/academy/data-processing) - [Scaling](https://scrapfly.io/academy/scaling) - [Walkthrough Summary](https://scrapfly.io/academy/walkthrough-summary) - [Scraper Blocking](https://scrapfly.io/academy/scraper-blocking) - [Proxies](https://scrapfly.io/academy/proxies) --- # Proxy Saver Protocols [ View as markdown ](https://scrapfly.io/?view=markdown) Copy for LLM Copy for LLM [ Open in ChatGPT ](https://chatgpt.com/?hints=search&prompt=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fproxy-saver%2Fprotocols%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Claude ](https://claude.ai/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fproxy-saver%2Fprotocols%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Perplexity ](https://www.perplexity.ai/search/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fproxy-saver%2Fprotocols%20so%20I%20can%20ask%20questions%20about%20it.) The Proxy Saver supports all standard and modern proxy protocols, ensuring compatibility with any browser or web scraping tool. Below are the details and specificities of each protocol's usage. > While all the provided examples account for the possibility that the Proxy Saver certificates may not be properly installed, the recommended approach is to [configure them correctly](https://scrapfly.io/docs/proxy-saver/certificates) rather than bypassing SSL errors. ## HTTP HTTP is the simplest proxy protocol and is universally compatible with all browsers and libraries. Below are some examples: - [Curl](#basic-use-curl-http) - [Playwright (js)](#scrape-http-proxy-playwright-js) - [Playwright (python)](#scrape-http-proxy-playwright-python) ``` curl \ --insecure \ --proxy http://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \ https://httpbin.dev/anything ``` ``` const {chromium} = require("playwright"); (async () => { try { const browser = await chromium.launch({ proxy: { server: 'http://proxy-saver.scrapfly.io:3333', username: 'proxyId-XXX', password: 'scp-live-XXX' } }); const context = await browser.newContext({ignoreHTTPSErrors: true}); const page = await context.newPage(); await page.goto("https://httpbin.dev/anything"); const html = await page.content(); console.log(html); await browser.close(); } catch (error) { console.error("An error occurred:", error); } })(); ``` ``` from playwright.sync_api import sync_playwright def main(): try: with sync_playwright() as p: browser = p.chromium.launch( proxy={ "server": "http://proxy-saver.scrapfly.io:3333", "username": "proxyId-XXX", "password": "scp-live-XXX", } ) context = browser.new_context(ignore_https_errors=True) page = context.new_page() page.goto("https://httpbin.dev/anything") html = page.content() print(html) except Exception as e: print("An error occurred:", e) finally: browser.close() if __name__ == "__main__": main() ``` ## HTTPS / HTTP2 Using HTTP as a proxy protocol is straightforward, but opting for HTTPS ensures that your credentials are securely encrypted from the very start of the CONNECT phase. By default, curl will use HTTP/1.1 over TLS unless you have a recent version that supports HTTP/2 and explicitly specify the necessary flags. Chrome, on the other hand, defaults to HTTP/2 but will gracefully downgrade to HTTP/1.1 over TLS if needed. The Proxy Saver natively supports HTTP/2 while also maintaining compatibility with HTTP/1.1 over TLS to accommodate the broadest range of browser implementations. - [Curl (HTTPS)](#basic-use-curl-https) - [Curl (HTTP2)](#basic-use-curl-http2) - [Playwright (js)](#scrape-https-proxy-playwright-js) - [Playwright (python)](#scrape-https-proxy-playwright-python) ``` curl \ --insecure --proxy-insecure \ --proxy https://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \ https://httpbin.dev/anything ``` ``` curl \ --insecure --proxy-insecure \ --proxy-http2 \ --proxy https://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \ https://httpbin.dev/anything ``` ``` const {chromium} = require("playwright"); (async () => { try { const browser = await chromium.launch({ proxy: { server: 'https://proxy-saver.scrapfly.io:3333', username: 'proxyId-XXX', password: 'scp-live-XXX' }, args: ['--ignore-certificate-errors'], }); const context = await browser.newContext({ignoreHTTPSErrors: true}); const page = await context.newPage(); await page.goto("https://httpbin.dev/anything"); const html = await page.content(); console.log(html); await browser.close(); } catch (error) { console.error("An error occurred:", error); } })(); ``` ``` from playwright.sync_api import sync_playwright def main(): try: with sync_playwright() as p: browser = p.chromium.launch( proxy={ "server": "https://proxy-saver.scrapfly.io:3333", "username": "proxyId-XXX", "password": "scp-live-XXX", }, args=["--ignore-certificate-errors"] ) context = browser.new_context(ignore_https_errors=True) page = context.new_page() page.goto("https://httpbin.dev/anything") html = page.content() print(html) except Exception as e: print("An error occurred:", e) finally: browser.close() if __name__ == "__main__": main() ``` ## SOCKS5 Socks5/h is another popular protocol used for HTTP proxying ### Socks5 auth mechanism For browsers and libraries that support native SOCKS5 authentication, the Proxy Saver does as well, ensuring seamless integration. - [Curl (SOCKS5 Auth)](#basic-use-curl-https) ``` curl \ --insecure \ --proxy socks5h://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \ https://httpbin.dev/anything ``` However, not all browsers support this mechanism. > While Chrome and Chromium do not support the SOCKS5 authentication mechanism (and have no plans to), Scrapfly offers a reliable workaround. For browsers and libraries that do not support SOCKS5 authentication natively, Scrapfly offers a simple workaround using a custom header. Simply include your authentication string in the `X-SAVER-AUTH` header. This custom authentication mechanism enables the use of the SOCKS5 protocol with any library that supports it. Below are some usage examples: - [Curl (X-SAVER-AUTH)](#basic-use-curl-socks5h-x-auth) - [Playwright (js)](#scrape-socks5-proxy-playwright-js) - [Playwright (python)](#scrape-socks5-proxy-playwright-python) ``` curl \ --insecure \ --proxy socks5h://proxy-saver.scrapfly.io:3333 \ --header "X-SAVER-AUTH: proxyId-XXX:scp-live-XXX" \ https://httpbin.dev/anything ``` ``` const { chromium } = require("playwright"); const xSaverAuth = { 'X-SAVER-AUTH': 'proxyId-XXX:scp-live-XXX'}; (async () => { try { const browser = await chromium.launch({ proxy: { server: 'socks5://proxy-saver.scrapfly.io:3333', } }); const context = await browser.newContext({ ignoreHTTPSErrors: true }); await context.setExtraHTTPHeaders(xSaverAuth); const page = await context.newPage(); await page.goto("https://httpbin.dev/anything"); const html = await page.content(); console.log(html); await browser.close(); } catch (error) { console.error("An error occurred:", error); } })(); ``` ``` from playwright.sync_api import sync_playwright def main(): try: with sync_playwright() as p: browser = p.chromium.launch( proxy={ "server": "socks5://proxy-saver.scrapfly.io:3333", } ) context = browser.new_context(ignore_https_errors=True) context.set_extra_http_headers({ "X-SAVER-AUTH": "proxyId-XXX:scp-live-XXX" }) page = context.new_page() page.goto("https://httpbin.dev/anything") html = page.content() print(html) except Exception as e: print("An error occurred:", e) finally: browser.close() if __name__ == "__main__": main() ```