# Scrapfly Documentation ## Table of Contents ### Dashboard - [Intro](https://scrapfly.io/docs) - [Project](https://scrapfly.io/docs/project) - [Account](https://scrapfly.io/docs/account) - [Workspace & Team](https://scrapfly.io/docs/workspace-and-team) - [Billing](https://scrapfly.io/docs/billing) ### Products #### MCP Server - [Getting Started](https://scrapfly.io/docs/mcp/getting-started) - [Tools & API Spec](https://scrapfly.io/docs/mcp/tools) - [Authentication](https://scrapfly.io/docs/mcp/authentication) - [Examples & Use Cases](https://scrapfly.io/docs/mcp/examples) - [FAQ](https://scrapfly.io/docs/mcp/faq) ##### Integrations - [Overview](https://scrapfly.io/docs/mcp/integrations) - [Claude Desktop](https://scrapfly.io/docs/mcp/integrations/claude-desktop) - [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code) - [ChatGPT](https://scrapfly.io/docs/mcp/integrations/chatgpt) - [Cursor](https://scrapfly.io/docs/mcp/integrations/cursor) - [Cline](https://scrapfly.io/docs/mcp/integrations/cline) - [Windsurf](https://scrapfly.io/docs/mcp/integrations/windsurf) - [Zed](https://scrapfly.io/docs/mcp/integrations/zed) - [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code) - [VS Code](https://scrapfly.io/docs/mcp/integrations/vscode) - [LangChain](https://scrapfly.io/docs/mcp/integrations/langchain) - [LlamaIndex](https://scrapfly.io/docs/mcp/integrations/llamaindex) - [CrewAI](https://scrapfly.io/docs/mcp/integrations/crewai) - [OpenAI](https://scrapfly.io/docs/mcp/integrations/openai) - [n8n](https://scrapfly.io/docs/mcp/integrations/n8n) - [Make](https://scrapfly.io/docs/mcp/integrations/make) - [Zapier](https://scrapfly.io/docs/mcp/integrations/zapier) - [Vapi AI](https://scrapfly.io/docs/mcp/integrations/vapi) - [Agent Builder](https://scrapfly.io/docs/mcp/integrations/agent-builder) - [Custom Client](https://scrapfly.io/docs/mcp/integrations/custom-client) #### Web Scraping API - [Getting Started](https://scrapfly.io/docs/scrape-api/getting-started) - [API Specification]() - [Monitoring](https://scrapfly.io/docs/monitoring) - [Customize Request](https://scrapfly.io/docs/scrape-api/custom) - [Debug](https://scrapfly.io/docs/scrape-api/debug) - [Anti Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - [Proxy](https://scrapfly.io/docs/scrape-api/proxy) - [Proxy Mode](https://scrapfly.io/docs/scrape-api/proxy-mode) - [Proxy Mode - Screaming Frog](https://scrapfly.io/docs/scrape-api/proxy-mode/screaming-frog) - [Proxy Mode - Apify](https://scrapfly.io/docs/scrape-api/proxy-mode/apify) - [(Auto) Data Extraction](https://scrapfly.io/docs/scrape-api/extraction) - [Javascript Rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - [Javascript Scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario) - [SSL](https://scrapfly.io/docs/scrape-api/ssl) - [DNS](https://scrapfly.io/docs/scrape-api/dns) - [Cache](https://scrapfly.io/docs/scrape-api/cache) - [Session](https://scrapfly.io/docs/scrape-api/session) - [Webhook](https://scrapfly.io/docs/scrape-api/webhook) - [Screenshot](https://scrapfly.io/docs/scrape-api/screenshot) - [Errors](https://scrapfly.io/docs/scrape-api/errors) - [Timeout](https://scrapfly.io/docs/scrape-api/understand-timeout) - [Throttling](https://scrapfly.io/docs/throttling) - [Troubleshoot](https://scrapfly.io/docs/scrape-api/troubleshoot) - [Billing](https://scrapfly.io/docs/scrape-api/billing) - [FAQ](https://scrapfly.io/docs/scrape-api/faq) #### Crawler API - [Getting Started](https://scrapfly.io/docs/crawler-api/getting-started) - [API Specification]() - [Retrieving Results](https://scrapfly.io/docs/crawler-api/results) - [WARC Format](https://scrapfly.io/docs/crawler-api/warc-format) - [Data Extraction](https://scrapfly.io/docs/crawler-api/extraction-rules) - [Webhook](https://scrapfly.io/docs/crawler-api/webhook) - [Billing](https://scrapfly.io/docs/crawler-api/billing) - [Errors](https://scrapfly.io/docs/crawler-api/errors) - [Troubleshoot](https://scrapfly.io/docs/crawler-api/troubleshoot) - [FAQ](https://scrapfly.io/docs/crawler-api/faq) #### Screenshot API - [Getting Started](https://scrapfly.io/docs/screenshot-api/getting-started) - [API Specification]() - [Accessibility Testing](https://scrapfly.io/docs/screenshot-api/accessibility) - [Webhook](https://scrapfly.io/docs/screenshot-api/webhook) - [Billing](https://scrapfly.io/docs/screenshot-api/billing) - [Errors](https://scrapfly.io/docs/screenshot-api/errors) #### Extraction API - [Getting Started](https://scrapfly.io/docs/extraction-api/getting-started) - [API Specification]() - [Rules Template](https://scrapfly.io/docs/extraction-api/rules-and-template) - [LLM Extraction](https://scrapfly.io/docs/extraction-api/llm-prompt) - [AI Auto Extraction](https://scrapfly.io/docs/extraction-api/automatic-ai) - [Webhook](https://scrapfly.io/docs/extraction-api/webhook) - [Billing](https://scrapfly.io/docs/extraction-api/billing) - [Errors](https://scrapfly.io/docs/extraction-api/errors) - [FAQ](https://scrapfly.io/docs/extraction-api/faq) #### Proxy Saver - [Getting Started](https://scrapfly.io/docs/proxy-saver/getting-started) - [Fingerprints](https://scrapfly.io/docs/proxy-saver/fingerprints) - [Optimizations](https://scrapfly.io/docs/proxy-saver/optimizations) - [SSL Certificates](https://scrapfly.io/docs/proxy-saver/certificates) - [Protocols](https://scrapfly.io/docs/proxy-saver/protocols) - [Pacfile](https://scrapfly.io/docs/proxy-saver/pacfile) - [Secure Credentials](https://scrapfly.io/docs/proxy-saver/security) - [Billing](https://scrapfly.io/docs/proxy-saver/billing) #### Cloud Browser API - [Getting Started](https://scrapfly.io/docs/cloud-browser-api/getting-started) - [Proxy & Geo-Targeting](https://scrapfly.io/docs/cloud-browser-api/proxy) - [Unblock API](https://scrapfly.io/docs/cloud-browser-api/unblock) - [File Downloads](https://scrapfly.io/docs/cloud-browser-api/file-downloads) - [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume) - [Human-in-the-Loop](https://scrapfly.io/docs/cloud-browser-api/human-in-the-loop) - [Debug Mode](https://scrapfly.io/docs/cloud-browser-api/debug-mode) - [Bring Your Own Proxy](https://scrapfly.io/docs/cloud-browser-api/bring-your-own-proxy) - [Browser Extensions](https://scrapfly.io/docs/cloud-browser-api/extensions) ##### Integrations - [Puppeteer](https://scrapfly.io/docs/cloud-browser-api/puppeteer) - [Playwright](https://scrapfly.io/docs/cloud-browser-api/playwright) - [Selenium](https://scrapfly.io/docs/cloud-browser-api/selenium) - [Vercel Agent Browser](https://scrapfly.io/docs/cloud-browser-api/agent-browser) - [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use) - [Stagehand](https://scrapfly.io/docs/cloud-browser-api/stagehand) - [Billing](https://scrapfly.io/docs/cloud-browser-api/billing) - [Errors](https://scrapfly.io/docs/cloud-browser-api/errors) ### Tools - [Antibot Detector](https://scrapfly.io/docs/tools/antibot-detector) ### SDK - [Golang](https://scrapfly.io/docs/sdk/golang) - [Python](https://scrapfly.io/docs/sdk/python) - [Rust](https://scrapfly.io/docs/sdk/rust) - [TypeScript](https://scrapfly.io/docs/sdk/typescript) - [Scrapy](https://scrapfly.io/docs/sdk/scrapy) ### Integrations - [Getting Started](https://scrapfly.io/docs/integration/getting-started) - [LangChain](https://scrapfly.io/docs/integration/langchain) - [LlamaIndex](https://scrapfly.io/docs/integration/llamaindex) - [CrewAI](https://scrapfly.io/docs/integration/crewai) - [Zapier](https://scrapfly.io/docs/integration/zapier) - [Make](https://scrapfly.io/docs/integration/make) - [n8n](https://scrapfly.io/docs/integration/n8n) ### Academy - [Overview](https://scrapfly.io/academy) - [Web Scraping Overview](https://scrapfly.io/academy/scraping-overview) - [Tools](https://scrapfly.io/academy/tools-overview) - [Reverse Engineering](https://scrapfly.io/academy/reverse-engineering) - [Static Scraping](https://scrapfly.io/academy/static-scraping) - [HTML Parsing](https://scrapfly.io/academy/html-parsing) - [Dynamic Scraping](https://scrapfly.io/academy/dynamic-scraping) - [Hidden API Scraping](https://scrapfly.io/academy/hidden-api-scraping) - [Headless Browsers](https://scrapfly.io/academy/headless-browsers) - [Hidden Web Data](https://scrapfly.io/academy/hidden-web-data) - [JSON Parsing](https://scrapfly.io/academy/json-parsing) - [Data Processing](https://scrapfly.io/academy/data-processing) - [Scaling](https://scrapfly.io/academy/scaling) - [Walkthrough Summary](https://scrapfly.io/academy/walkthrough-summary) - [Scraper Blocking](https://scrapfly.io/academy/scraper-blocking) - [Proxies](https://scrapfly.io/academy/proxies) --- # Bring Your Own Proxy (BYOP) [ View as markdown ](https://scrapfly.io/?view=markdown) Copy for LLM Copy for LLM [ Open in ChatGPT ](https://chatgpt.com/?hints=search&prompt=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fcloud-browser-api%2Fbring-your-own-proxy%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Claude ](https://claude.ai/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fcloud-browser-api%2Fbring-your-own-proxy%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Perplexity ](https://www.perplexity.ai/search/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fcloud-browser-api%2Fbring-your-own-proxy%20so%20I%20can%20ask%20questions%20about%20it.) **Bring Your Own Proxy** allows you to use your own proxy servers with Cloud Browser instead of Scrapfly's managed proxy pools. This is ideal for customers who have existing proxy infrastructure or specific proxy requirements. **Custom Plan Feature:** BYOP is exclusively available for Custom plan subscribers. [Request a custom plan](https://scrapfly.io/dashboard/billing/plan/custom) or contact . **Success Rate Consideration:** When using BYOP, Scrapfly does not control the IP quality, reputation, or coherence of your proxy. Blocking issues and lower success rates caused by your proxy infrastructure are outside Scrapfly's support scope. Our managed proxy pools include intelligent rotation, automatic failover, and target-specific tuning that external proxies cannot replicate. If you experience higher block rates, consider switching to [Scrapfly's built-in proxy options](https://scrapfly.io/docs/cloud-browser-api/proxy). ## Overview With BYOP, you provide your own proxy URL and Cloud Browser routes all browser traffic through your proxy. This gives you: - **Full Control:** Use proxies from any provider with your own authentication - **Existing Infrastructure:** Leverage your current proxy setup without changes - **No Bandwidth Billing:** Since you're using your own proxy, Scrapfly does not bill for bandwidth - **Custom Sessions:** Use session/sticky proxy features from your provider ## Proxy URL Format The BYOP proxy URL must follow this format: ``` {protocol}://{username}:{password}@{host}:{port} ``` **URL Encoding:** Remember to URL-encode the `byop_proxy` parameter if it contains special characters in the username or password. ### Supported Protocols | Protocol | Description | Example | |---|---|---| | `http` | HTTP proxy (most common) | `http://user:pass@proxy.your-provider.com:8080` | | `https` | HTTPS proxy with TLS | `https://user:pass@proxy.your-provider.com:443` | | `socks5` | SOCKS5 proxy (TCP only) | `socks5://user:pass@proxy.your-provider.com:1080` | | `socks5h` | SOCKS5 proxy with remote DNS resolution | `socks5h://user:pass@proxy.your-provider.com:1080` | | `socks5+udp` | SOCKS5 proxy with UDP support | `socks5+udp://user:pass@proxy.your-provider.com:1080` | | `socks5h+udp` | SOCKS5 proxy with remote DNS and UDP support | `socks5h+udp://user:pass@proxy.your-provider.com:1080` | **SOCKS5 UDP & Authentication:** Use `socks5+udp://` or `socks5h+udp://` to enable [UDP ASSOCIATE (RFC 1928)](https://datatracker.ietf.org/doc/html/rfc1928#section-4) on your SOCKS5 proxy. This allows the browser to use **QUIC/HTTP/3** through the proxy, which improves performance and reduces detection by anti-bot systems that expect real browsers to use HTTP/3. Cloud Browser is powered by Scrapium, our custom-built browser engine based on Chromium. Unlike genuine Chromium, which does not support SOCKS5 authentication or UDP ASSOCIATE, Scrapium natively supports both **SOCKS5 authentication** and **SOCKS5 UDP ASSOCIATE**, enabling full QUIC/HTTP/3 traffic through authenticated SOCKS5 proxies. Scrapium also proxifies **WebRTC/STUN** traffic through the SOCKS5 proxy to prevent IP leaks. Note that some proxy providers may block the UDP ports used by STUN. ## Usage Pass your proxy URL via the `byop_proxy` parameter when creating a Cloud Browser session. Replace the placeholder proxy URL in the examples below with your actual proxy URL before running the code. The Scrapfly SDKs expose `byop_proxy` directly on the browser config object, which is the recommended way to use BYOP across all languages. ### Python SDK Install: `pip install scrapfly-sdk[playwright]`. The `byop_proxy` argument was added in Python SDK version **0.8.25**. ``` from scrapfly import ScrapflyClient, BrowserConfig from playwright.sync_api import sync_playwright client = ScrapflyClient(key='{{ YOUR_API_KEY }}') # Bring Your Own Proxy URL. # Format: {protocol}://{user}:{pass}@{host}:{port} # Supported protocols: # http, https, socks5, socks5h, socks5+udp, socks5h+udp # The +udp variants enable HTTP/3 (QUIC) via SOCKS5 UDP ASSOCIATE, # only with proxy providers that implement RFC 1928 UDP ASSOCIATE. config = BrowserConfig( byop_proxy='http://user:pass@proxy.your-provider.com:8080', # replace with your proxy os='linux', ) with sync_playwright() as p: browser = p.chromium.connect_over_cdp(client.cloud_browser(config)) context = browser.contexts[0] page = context.pages[0] if context.pages else context.new_page() page.goto('https://web-scraping.dev/products') # The page title above was fetched through YOUR BYOP proxy. print('Page title via BYOP proxy:', page.title()) browser.close() ``` ### JavaScript / TypeScript SDK Install: `npm install scrapfly-sdk playwright`. The `byop_proxy` option was added in `scrapfly-sdk` version **0.7.4**. ``` import { ScrapflyClient, BrowserConfig } from 'scrapfly-sdk'; import { chromium } from 'playwright'; const client = new ScrapflyClient({ key: '{{ YOUR_API_KEY }}' }); // Bring Your Own Proxy URL. // Format: {protocol}://{user}:{pass}@{host}:{port} // Supported protocols: // http, https, socks5, socks5h, socks5+udp, socks5h+udp // The +udp variants enable HTTP/3 (QUIC) via SOCKS5 UDP ASSOCIATE, // only with proxy providers that implement RFC 1928 UDP ASSOCIATE. const config = new BrowserConfig({ byop_proxy: 'http://user:pass@proxy.your-provider.com:8080', // replace with your proxy os: 'linux', }); const browser = await chromium.connectOverCDP(client.cloudBrowser(config)); const context = browser.contexts()[0]; const page = context.pages()[0] || (await context.newPage()); await page.goto('https://web-scraping.dev/products'); // The page title above was fetched through YOUR BYOP proxy. console.log('Page title via BYOP proxy:', await page.title()); await browser.close(); ``` ### Go SDK Install: `go get github.com/scrapfly/go-scrapfly`. The `BYOPProxy` field on `CloudBrowserConfig` was added in version **v0.5.0**. The example below uses [chromedp](https://github.com/chromedp/chromedp) as the CDP client; [rod](https://github.com/go-rod/rod) and playwright-go work the same way. ``` package main import ( "context" "fmt" "log" "github.com/chromedp/chromedp" scrapfly "github.com/scrapfly/go-scrapfly" ) func main() { client, err := scrapfly.New("{{ YOUR_API_KEY }}") if err != nil { log.Fatalf("failed to create client: %v", err) } // Bring Your Own Proxy URL. // Format: {protocol}://{user}:{pass}@{host}:{port} // Supported protocols: // http, https, socks5, socks5h, socks5+udp, socks5h+udp // The +udp variants enable HTTP/3 (QUIC) via SOCKS5 UDP ASSOCIATE, // only with proxy providers that implement RFC 1928 UDP ASSOCIATE. wsURL, err := client.CloudBrowser(&scrapfly.CloudBrowserConfig{ BYOPProxy: "http://user:pass@proxy.your-provider.com:8080", // replace with your proxy OS: scrapfly.CloudBrowserOSLinux, }) if err != nil { log.Fatalf("invalid cloud-browser config: %v", err) } // Connect to the Cloud Browser via chromedp's remote allocator. allocCtx, cancelAlloc := chromedp.NewRemoteAllocator(context.Background(), wsURL) defer cancelAlloc() taskCtx, cancelTask := chromedp.NewContext(allocCtx) defer cancelTask() var pageTitle string if err := chromedp.Run(taskCtx, chromedp.Navigate("https://web-scraping.dev/products"), chromedp.Title(&pageTitle), ); err != nil { log.Fatalf("chromedp run failed: %v", err) } // The page title above was fetched through YOUR BYOP proxy. fmt.Println("Page title via BYOP proxy:", pageTitle) } ``` ### Raw WebSocket URL (any language) If you're not using a Scrapfly SDK, build the WebSocket URL manually and connect with any CDP client. The `byop_proxy` value must be URL-encoded so the `+`, `:`, `@` and `/` characters survive query-string parsing on the server side. Most HTTP libraries do this for you automatically. ```bash # Replace YOUR_PROXY_URL with your actual proxy URL. # Special characters (+, :, @, /) must be URL-encoded. WS_URL="wss://browser.scrapfly.io?api_key={{ YOUR_API_KEY }}&os=linux&byop_proxy=http%3A%2F%2Fuser%3Apass%40proxy.your-provider.com%3A8080" ``` ### Supported proxy protocols The `byop_proxy` URL must use one of these schemes. Cloud Browser will reject any other protocol with `ERR::BROWSER::BYOP_INVALID_PROXY_FORMAT`. | Scheme | Use case | |---|---| | `http` | Plain HTTP proxy (most common, supported by all providers) | | `https` | HTTP proxy over TLS | | `socks5` | SOCKS5 proxy with local DNS resolution (browser resolves the hostname) | | `socks5h` | SOCKS5 proxy with remote DNS resolution (proxy resolves the hostname — prevents DNS leaks) | | `socks5+udp` | SOCKS5 with [UDP ASSOCIATE](https://datatracker.ietf.org/doc/html/rfc1928#section-7) — enables **HTTP/3 (QUIC)** through the proxy. Most providers do not implement this; check with yours before using it. | | `socks5h+udp` | SOCKS5h with UDP ASSOCIATE — remote DNS *and* HTTP/3 support. Same provider caveat as `socks5+udp`. | **HTTP/3 caveat:** The `+udp` variants only work with proxy providers that implement SOCKS5 UDP ASSOCIATE (RFC 1928 §7). If your provider does not, QUIC packets are silently dropped and pages may load slowly, partially, or not at all — without any clear error. When in doubt, use plain `socks5h://`. ## Your Responsibilities **Important:** When using BYOP, you are responsible for: - **Proxy Configuration:** Ensuring your proxy is correctly configured with proper authentication, session settings, and any provider-specific options (e.g., sticky sessions, rotation rules). - **Proxy Type Selection:** Choosing the appropriate proxy type (datacenter, residential, mobile) for your target websites. Some sites may block certain proxy types. - **Geographic Targeting:** Configuring the correct country/region in your proxy settings if geo-targeting is required. - **Proxy Availability:** Ensuring your proxy server is accessible and has sufficient capacity for your workload. - **Error Handling:** Handling any connection failures or timeouts from your proxy provider. ## Combining with Other Options BYOP can be combined with other Cloud Browser parameters: | Parameter | Works with BYOP? | Notes | |---|---|---| | `os` | Yes | Browser fingerprint OS still applies | | `session` | Yes | Session persistence works normally | | `timeout` | Yes | Session timeout works normally | | `country` | Ignored | Country is determined by your proxy, not Scrapfly | | `proxy_pool` | Ignored | Your BYOP proxy is used instead | ## Billing With BYOP: - **Runtime:** Billed normally based on session duration - **Bandwidth:** **Not billed** - you're using your own proxy infrastructure - **API Credits:** Calculated based on runtime only See [Cloud Browser Billing](https://scrapfly.io/docs/cloud-browser-api/billing) for detailed pricing information. ## Error Handling ### BYOP-Specific Errors | Error Code | Description | Solution | |---|---|---| | `ERR::BROWSER::BYOP_CUSTOM_PLAN_REQUIRED` | BYOP requires Custom plan subscription | [Contact us](https://scrapfly.io/contact) to upgrade your plan | | `ERR::BROWSER::BYOP_INVALID_PROXY_FORMAT` | Invalid proxy URL format | Ensure URL follows `{protocol}://{user}:{pass}@{host}:{port}` | | `503: BYOP proxy connectivity test failed` | Could not connect to your proxy | Verify proxy is accessible and credentials are correct | ## Troubleshooting ### Proxy Connection Failed If you receive a proxy connectivity error: 1. Verify your proxy is accessible from the internet (not behind a firewall) 2. Check username and password are correct 3. Ensure the proxy port is open and listening 4. Test your proxy locally with `curl --proxy` before using with Cloud Browser ### Unexpected IP Address If the browser's IP doesn't match your expected proxy exit IP: 1. Verify your proxy provider's session/rotation settings 2. Check if your proxy requires specific headers or configuration 3. Some proxies use different exit IPs for different protocols (HTTP vs HTTPS) ### Target Website Blocks If target websites block your requests: 1. Ensure you're using the appropriate proxy type (residential proxies for sensitive targets) 2. Check if your proxy provider's IPs are on blocklists 3. Consider using sticky sessions for consistent IP identity