# Scrapfly Documentation ## Table of Contents ### Dashboard - [Intro](https://scrapfly.io/docs) - [Project](https://scrapfly.io/docs/project) - [Account](https://scrapfly.io/docs/account) - [Workspace & Team](https://scrapfly.io/docs/workspace-and-team) - [Billing](https://scrapfly.io/docs/billing) ### Products #### MCP Server - [Getting Started](https://scrapfly.io/docs/mcp/getting-started) - [Tools & API Spec](https://scrapfly.io/docs/mcp/tools) - [Authentication](https://scrapfly.io/docs/mcp/authentication) - [Examples & Use Cases](https://scrapfly.io/docs/mcp/examples) - [FAQ](https://scrapfly.io/docs/mcp/faq) ##### Integrations - [Overview](https://scrapfly.io/docs/mcp/integrations) - [Claude Desktop](https://scrapfly.io/docs/mcp/integrations/claude-desktop) - [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code) - [ChatGPT](https://scrapfly.io/docs/mcp/integrations/chatgpt) - [Cursor](https://scrapfly.io/docs/mcp/integrations/cursor) - [Cline](https://scrapfly.io/docs/mcp/integrations/cline) - [Windsurf](https://scrapfly.io/docs/mcp/integrations/windsurf) - [Zed](https://scrapfly.io/docs/mcp/integrations/zed) - [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code) - [VS Code](https://scrapfly.io/docs/mcp/integrations/vscode) - [LangChain](https://scrapfly.io/docs/mcp/integrations/langchain) - [LlamaIndex](https://scrapfly.io/docs/mcp/integrations/llamaindex) - [CrewAI](https://scrapfly.io/docs/mcp/integrations/crewai) - [OpenAI](https://scrapfly.io/docs/mcp/integrations/openai) - [n8n](https://scrapfly.io/docs/mcp/integrations/n8n) - [Make](https://scrapfly.io/docs/mcp/integrations/make) - [Zapier](https://scrapfly.io/docs/mcp/integrations/zapier) - [Vapi AI](https://scrapfly.io/docs/mcp/integrations/vapi) - [Agent Builder](https://scrapfly.io/docs/mcp/integrations/agent-builder) - [Custom Client](https://scrapfly.io/docs/mcp/integrations/custom-client) #### Web Scraping API - [Getting Started](https://scrapfly.io/docs/scrape-api/getting-started) - [API Specification]() - [Monitoring](https://scrapfly.io/docs/monitoring) - [Customize Request](https://scrapfly.io/docs/scrape-api/custom) - [Debug](https://scrapfly.io/docs/scrape-api/debug) - [Anti Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - [Proxy](https://scrapfly.io/docs/scrape-api/proxy) - [Proxy Mode](https://scrapfly.io/docs/scrape-api/proxy-mode) - [Proxy Mode - Screaming Frog](https://scrapfly.io/docs/scrape-api/proxy-mode/screaming-frog) - [Proxy Mode - Apify](https://scrapfly.io/docs/scrape-api/proxy-mode/apify) - [(Auto) Data Extraction](https://scrapfly.io/docs/scrape-api/extraction) - [Javascript Rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - [Javascript Scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario) - [SSL](https://scrapfly.io/docs/scrape-api/ssl) - [DNS](https://scrapfly.io/docs/scrape-api/dns) - [Cache](https://scrapfly.io/docs/scrape-api/cache) - [Session](https://scrapfly.io/docs/scrape-api/session) - [Webhook](https://scrapfly.io/docs/scrape-api/webhook) - [Screenshot](https://scrapfly.io/docs/scrape-api/screenshot) - [Errors](https://scrapfly.io/docs/scrape-api/errors) - [Timeout](https://scrapfly.io/docs/scrape-api/understand-timeout) - [Throttling](https://scrapfly.io/docs/throttling) - [Troubleshoot](https://scrapfly.io/docs/scrape-api/troubleshoot) - [Billing](https://scrapfly.io/docs/scrape-api/billing) - [FAQ](https://scrapfly.io/docs/scrape-api/faq) #### Crawler API - [Getting Started](https://scrapfly.io/docs/crawler-api/getting-started) - [API Specification]() - [Retrieving Results](https://scrapfly.io/docs/crawler-api/results) - [WARC Format](https://scrapfly.io/docs/crawler-api/warc-format) - [Data Extraction](https://scrapfly.io/docs/crawler-api/extraction-rules) - [Webhook](https://scrapfly.io/docs/crawler-api/webhook) - [Billing](https://scrapfly.io/docs/crawler-api/billing) - [Errors](https://scrapfly.io/docs/crawler-api/errors) - [Troubleshoot](https://scrapfly.io/docs/crawler-api/troubleshoot) - [FAQ](https://scrapfly.io/docs/crawler-api/faq) #### Screenshot API - [Getting Started](https://scrapfly.io/docs/screenshot-api/getting-started) - [API Specification]() - [Accessibility Testing](https://scrapfly.io/docs/screenshot-api/accessibility) - [Webhook](https://scrapfly.io/docs/screenshot-api/webhook) - [Billing](https://scrapfly.io/docs/screenshot-api/billing) - [Errors](https://scrapfly.io/docs/screenshot-api/errors) #### Extraction API - [Getting Started](https://scrapfly.io/docs/extraction-api/getting-started) - [API Specification]() - [Rules Template](https://scrapfly.io/docs/extraction-api/rules-and-template) - [LLM Extraction](https://scrapfly.io/docs/extraction-api/llm-prompt) - [AI Auto Extraction](https://scrapfly.io/docs/extraction-api/automatic-ai) - [Webhook](https://scrapfly.io/docs/extraction-api/webhook) - [Billing](https://scrapfly.io/docs/extraction-api/billing) - [Errors](https://scrapfly.io/docs/extraction-api/errors) - [FAQ](https://scrapfly.io/docs/extraction-api/faq) #### Proxy Saver - [Getting Started](https://scrapfly.io/docs/proxy-saver/getting-started) - [Fingerprints](https://scrapfly.io/docs/proxy-saver/fingerprints) - [Optimizations](https://scrapfly.io/docs/proxy-saver/optimizations) - [SSL Certificates](https://scrapfly.io/docs/proxy-saver/certificates) - [Protocols](https://scrapfly.io/docs/proxy-saver/protocols) - [Pacfile](https://scrapfly.io/docs/proxy-saver/pacfile) - [Secure Credentials](https://scrapfly.io/docs/proxy-saver/security) - [Billing](https://scrapfly.io/docs/proxy-saver/billing) #### Cloud Browser API - [Getting Started](https://scrapfly.io/docs/cloud-browser-api/getting-started) - [Proxy & Geo-Targeting](https://scrapfly.io/docs/cloud-browser-api/proxy) - [Unblock API](https://scrapfly.io/docs/cloud-browser-api/unblock) - [File Downloads](https://scrapfly.io/docs/cloud-browser-api/file-downloads) - [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume) - [Human-in-the-Loop](https://scrapfly.io/docs/cloud-browser-api/human-in-the-loop) - [Debug Mode](https://scrapfly.io/docs/cloud-browser-api/debug-mode) - [Bring Your Own Proxy](https://scrapfly.io/docs/cloud-browser-api/bring-your-own-proxy) - [Browser Extensions](https://scrapfly.io/docs/cloud-browser-api/extensions) ##### Integrations - [Puppeteer](https://scrapfly.io/docs/cloud-browser-api/puppeteer) - [Playwright](https://scrapfly.io/docs/cloud-browser-api/playwright) - [Selenium](https://scrapfly.io/docs/cloud-browser-api/selenium) - [Vercel Agent Browser](https://scrapfly.io/docs/cloud-browser-api/agent-browser) - [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use) - [Stagehand](https://scrapfly.io/docs/cloud-browser-api/stagehand) - [Billing](https://scrapfly.io/docs/cloud-browser-api/billing) - [Errors](https://scrapfly.io/docs/cloud-browser-api/errors) ### Tools - [Antibot Detector](https://scrapfly.io/docs/tools/antibot-detector) ### SDK - [Golang](https://scrapfly.io/docs/sdk/golang) - [Python](https://scrapfly.io/docs/sdk/python) - [TypeScript](https://scrapfly.io/docs/sdk/typescript) - [Scrapy](https://scrapfly.io/docs/sdk/scrapy) ### Integrations - [Getting Started](https://scrapfly.io/docs/integration/getting-started) - [LangChain](https://scrapfly.io/docs/integration/langchain) - [LlamaIndex](https://scrapfly.io/docs/integration/llamaindex) - [CrewAI](https://scrapfly.io/docs/integration/crewai) - [Zapier](https://scrapfly.io/docs/integration/zapier) - [Make](https://scrapfly.io/docs/integration/make) - [n8n](https://scrapfly.io/docs/integration/n8n) ### Academy - [Overview](https://scrapfly.io/academy) - [Web Scraping Overview](https://scrapfly.io/academy/scraping-overview) - [Tools](https://scrapfly.io/academy/tools-overview) - [Reverse Engineering](https://scrapfly.io/academy/reverse-engineering) - [Static Scraping](https://scrapfly.io/academy/static-scraping) - [HTML Parsing](https://scrapfly.io/academy/html-parsing) - [Dynamic Scraping](https://scrapfly.io/academy/dynamic-scraping) - [Hidden API Scraping](https://scrapfly.io/academy/hidden-api-scraping) - [Headless Browsers](https://scrapfly.io/academy/headless-browsers) - [Hidden Web Data](https://scrapfly.io/academy/hidden-web-data) - [JSON Parsing](https://scrapfly.io/academy/json-parsing) - [Data Processing](https://scrapfly.io/academy/data-processing) - [Scaling](https://scrapfly.io/academy/scaling) - [Walkthrough Summary](https://scrapfly.io/academy/walkthrough-summary) - [Scraper Blocking](https://scrapfly.io/academy/scraper-blocking) - [Proxies](https://scrapfly.io/academy/proxies) --- # Proxy [ View as markdown ](https://scrapfly.io/?view=markdown) Copy for LLM Copy for LLM [ Open in ChatGPT ](https://chatgpt.com/?hints=search&prompt=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fscrape-api%2Fproxy%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Claude ](https://claude.ai/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fscrape-api%2Fproxy%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Perplexity ](https://www.perplexity.ai/search/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fscrape-api%2Fproxy%20so%20I%20can%20ask%20questions%20about%20it.) Proxies are at the core of every modern web scraper as they allow traffic distribution through multiple connection agents. In other words, proxies in web scraping allow to: - Change origin country to access geographically locked content. - Bypass scraper blocking and anti-bot detection solutions. - Bypass scraper throttling. All Scrapfly requests go through proxies and are automatically configured for the best scraping experience. However, each scrapfly API call can be additionally configured manually via [proxy\_pool](https://scrapfly.io/docs/scrape-api/getting-started#api_param_proxy_pool) and [country](https://scrapfly.io/docs/scrape-api/getting-started#api_param_country) parameters. [See Your Proxy Dashboard](https://scrapfly.io/dashboard/proxy/scraper-api) ## What are Proxies? Each address is associated with an Autonomous System Number ([ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet))) and linked to a proxy. This is how anti-bot solutions track the origin of the IP and limit requests per IP. Our proxy pool rotates IPs, cools them down, and excludes underperforming proxies to increase success rates. We offer two public proxy pools by default, with the option to set up a custom pool. We manage and monitor our proxy pool for optimal performance, with both datacenter and residential proxies available. Different kind of proxies exists, by types of IP address issuer: - **Datacenter**: Cheap but are detected by advanced anti-bot solutions. - **Residential**: More expensive than **datacenter** and often required to bypass most anti-bot solutions. ### Why are Proxies Needed? Anti-bot solutions rate-limit the number of requests per connecting [IP address](https://en.wikipedia.org/wiki/IP_address). Meaning, to scrape more than a page or two scrapers need to distribute traffic through multiple IP addresses. This limit is why we need to manage the proxy pool, rotate IPs, and cool them to increase the success rate. ## Available Proxy Pools A proxy pool represents a group of proxies. By default, proxies are grouped by network type like **datacenter**, **residential**, **4g** etc. One of the main challenges with proxies is the reliability, it's complex to maintain a network of healthy proxies and routing the traffic to them. You don't have to manage proxies yourself - simply select a proxy pool and we'll route the traffic using a healthy proxy meeting your requirements. All available proxy pools are listed in the [proxy section on your dashboard](https://scrapfly.io/dashboard/proxy/scraper-api). Each proxy pool has its own cost (mostly due to network type). By default, these public pools are available: - **Public Datacenter Pool**: `public_datacenter_pool` - 1 API Credits will be counted - **Public Residential Pool**: `public_residential_pool` - 25 API Credits will be counted For example, to use residential proxies configure requests with the `proxy_pool=public_residential_pool` parameter: [Curl](#player-3b22b1) [HTTP](#http-3b22b1) [ ](https://scrapfly.io/login "Sign in to try from the API player") ``` curl -G \ --request "GET" \ --url "https://api.scrapfly.io/scrape" \ --data-urlencode "proxy_pool=public_residential_pool" \ --data-urlencode "key=__API_KEY__" \ --data-urlencode "url=https://httpbin.dev/anything" ``` ``` https://api.scrapfly.io/scrape?proxy_pool=public_residential_pool&key=&url=https%3A%2F%2Fhttpbin.dev%2Fanything ``` ## Geo Targeting All Scrapfly scrape requests go through a proxy meaning geo-targeting is available at all times. By default, Scrapfly will smartly select a random proxy from the selected pool for the highest success probability. To set the desired country, the [country](https://scrapfly.io/docs/scrape-api/getting-started#api_param_country) parameter can be used, and the available country options depend on the selected [proxy\_pool](https://scrapfly.io/docs/scrape-api/getting-started#api_param_proxy_pool): - [Public Datacenter Pool](#public_datacenter_pool) - [Public Residential Pool](#public_residential_pool) **API Pool Name:** `public_datacenter_pool` - United Arab Emirates : `ae` - Albania : `al` - Armenia : `am` - Argentina : `ar` - Austria : `at` - Australia : `au` - Bangladesh : `bd` - Belgium : `be` - Bulgaria : `bg` - Bolivia : `bo` - Brazil : `br` - Bahamas : `bs` - Belarus : `by` - Canada : `ca` - Switzerland : `ch` - Chile : `cl` - China : `cn` - Colombia : `co` - Costa Rica : `cr` - Cyprus : `cy` - Czechia : `cz` - Germany : `de` - Denmark : `dk` - Ecuador : `ec` - Estonia : `ee` - Egypt : `eg` - Spain : `es` - Finland : `fi` - France : `fr` - United Kingdom : `gb` - Georgia : `ge` - Greece : `gr` - Hong Kong SAR China : `hk` - Honduras : `hn` - Croatia : `hr` - Hungary : `hu` - Indonesia : `id` - Ireland : `ie` - Israel : `il` - India : `in` - Iraq : `iq` - Iceland : `is` - Italy : `it` - Jamaica : `jm` - Jordan : `jo` - Japan : `jp` - Cambodia : `kh` - South Korea : `kr` - Lithuania : `lt` - Latvia : `lv` - Morocco : `ma` - Moldova : `md` - Madagascar : `mg` - Mongolia : `mn` - Mexico : `mx` - Malaysia : `my` - Nigeria : `ng` - Netherlands : `nl` - Norway : `no` - New Zealand : `nz` - Panama : `pa` - Peru : `pe` - Philippines : `ph` - Pakistan : `pk` - Poland : `pl` - Portugal : `pt` - Romania : `ro` - Russia : `ru` - Saudi Arabia : `sa` - Sweden : `se` - Singapore : `sg` - Slovakia : `sk` - Thailand : `th` - Turkmenistan : `tm` - Tunisia : `tn` - Türkiye : `tr` - Taiwan : `tw` - Ukraine : `ua` - United States : `us` - Uzbekistan : `uz` - Venezuela : `ve` - British Virgin Islands : `vg` - Vietnam : `vn` - South Africa : `za` **API Pool Name:** `public_residential_pool` - United Arab Emirates : `ae` - Albania : `al` - Armenia : `am` - Argentina : `ar` - Austria : `at` - Australia : `au` - Bangladesh : `bd` - Belgium : `be` - Bulgaria : `bg` - Bolivia : `bo` - Brazil : `br` - Bahamas : `bs` - Belarus : `by` - Canada : `ca` - Switzerland : `ch` - Chile : `cl` - China : `cn` - Colombia : `co` - Costa Rica : `cr` - Cyprus : `cy` - Czechia : `cz` - Germany : `de` - Denmark : `dk` - Ecuador : `ec` - Estonia : `ee` - Egypt : `eg` - Spain : `es` - Finland : `fi` - France : `fr` - United Kingdom : `gb` - Georgia : `ge` - Greece : `gr` - Hong Kong SAR China : `hk` - Honduras : `hn` - Croatia : `hr` - Hungary : `hu` - Indonesia : `id` - Ireland : `ie` - Israel : `il` - India : `in` - Iraq : `iq` - Iceland : `is` - Italy : `it` - Jamaica : `jm` - Jordan : `jo` - Japan : `jp` - Cambodia : `kh` - South Korea : `kr` - Lithuania : `lt` - Latvia : `lv` - Morocco : `ma` - Moldova : `md` - Madagascar : `mg` - Mongolia : `mn` - Mexico : `mx` - Malaysia : `my` - Nigeria : `ng` - Netherlands : `nl` - Norway : `no` - New Zealand : `nz` - Panama : `pa` - Peru : `pe` - Philippines : `ph` - Pakistan : `pk` - Poland : `pl` - Portugal : `pt` - Romania : `ro` - Russia : `ru` - Saudi Arabia : `sa` - Sweden : `se` - Singapore : `sg` - Slovakia : `sk` - Thailand : `th` - Turkmenistan : `tm` - Tunisia : `tn` - Türkiye : `tr` - Taiwan : `tw` - Ukraine : `ua` - United States : `us` - Uzbekistan : `uz` - Venezuela : `ve` - British Virgin Islands : `vg` - Vietnam : `vn` - South Africa : `za` For example, to target Canada configure requests with the `country=CA` parameter: [Curl](#player-c50343) [HTTP](#http-c50343) [ ](https://scrapfly.io/login "Sign in to try from the API player") ``` curl -G \ --request "GET" \ --url "https://api.scrapfly.io/scrape" \ --data-urlencode "country=ca" \ --data-urlencode "key=__API_KEY__" \ --data-urlencode "url=https://httpbin.dev/anything" ``` ``` https://api.scrapfly.io/scrape?country=ca&key=&url=https%3A%2F%2Fhttpbin.dev%2Fanything ``` *You can also declare multiple country `ca,us` and you can also exclude some `-mx,-au`. To know more, you can check the [API parameters country](https://scrapfly.io/docs/scrape-api/getting-started#api_param_country) section.* ## Related Errors All related errors are listed below. You can see full description and example of error response on [Errors section](https://scrapfly.io/docs/scrape-api/errors#proxy) - [ERR::PROXY::POOL\_NOT\_AVAILABLE\_FOR\_TARGET](https://scrapfly.io/docs/scrape-api/error/ERR::PROXY::POOL_NOT_AVAILABLE_FOR_TARGET "The desired proxy pool is not available for the given domain - mostly well known protected domain which require at least residential networks") - The desired proxy pool is not available for the given domain - mostly well known protected domain which require at least residential networks - [ERR::PROXY::POOL\_NOT\_FOUND](https://scrapfly.io/docs/scrape-api/error/ERR::PROXY::POOL_NOT_FOUND "Provided Proxy Pool Name do not exists") - Provided Proxy Pool Name do not exists - [ERR::PROXY::POOL\_UNAVAILABLE\_COUNTRY](https://scrapfly.io/docs/scrape-api/error/ERR::PROXY::POOL_UNAVAILABLE_COUNTRY "Country not available for given proxy pool") - Country not available for given proxy pool - [ERR::PROXY::RESOURCES\_SATURATION](https://scrapfly.io/docs/scrape-api/error/ERR::PROXY::RESOURCES_SATURATION "Proxy are saturated for the desired country, you can try on other countries. They will come back as soon as possible") - Proxy are saturated for the desired country, you can try on other countries. They will come back as soon as possible - [ERR::PROXY::TIMEOUT](https://scrapfly.io/docs/scrape-api/error/ERR::PROXY::TIMEOUT "Proxy connection or website was too slow and timeout") - Proxy connection or website was too slow and timeout - [ERR::PROXY::UNAVAILABLE](https://scrapfly.io/docs/scrape-api/error/ERR::PROXY::UNAVAILABLE "Proxy is unavailable - The domain (mainly gov website) is restricted, You are using session feature and the proxy is unreachable at the moment") - Proxy is unavailable - The domain (mainly gov website) is restricted, You are using session feature and the proxy is unreachable at the moment ## Pricing Each call using the residential proxy pool will count for 25 Scrape API Credits. API responses contain a header `X-Scrapfly-Api-Cost` which indicate the total billed amount.