[Blog](https://scrapfly.io/blog) / [proxies](https://scrapfly.io/blog/tag/proxies) / [CloudProxy Web Scraping: A 2026 Reality Check](https://scrapfly.io/blog/posts/how-to-use-cloudproxy-for-web-scraping) # CloudProxy Web Scraping: A 2026 Reality Check by [Mayada Shaaban](https://scrapfly.io/blog/author/mayada-shaaban-90143e67) Aug 10, 2026 20 min read [\#proxies](https://scrapfly.io/blog/tag/proxies) [\#tools](https://scrapfly.io/blog/tag/tools) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping&text=CloudProxy%20Web%20Scraping%3A%20A%202026%20Reality%20Check "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-use-cloudproxy-for-web-scraping) **Web Scraping API — Anti-Bot Bypass**Bypass any anti-scraper system and automatically resolve JavaScript and fingerprint challenges. [ Learn More ](https://scrapfly.io/products/web-scraping-api#features) [ Docs ](https://scrapfly.io/docs/scrape-api/getting-started#features) A scraper can run perfectly on your laptop and return 403s the moment it moves to a cloud VM. One possible cause is a rule matching the provider's network. When that is the rule, starting another instance in the same network changes nothing. CloudProxy sits in that gap. It provisions and health-checks proxy VMs across cloud accounts you already own. This article gives you a working setup, a monthly cost you can calculate, and the case for running your own pool. [The Complete Guide To Using Proxies For Web ScrapingIntroduction to proxy usage in web scraping. What types of proxies are there? How to evaluate proxy providers and avoid common issues.](https://scrapfly.io/blog/posts/introduction-to-proxies-in-web-scraping) ## Key Takeaways - **Control plane, not gateway**: your scraper connects to the VM, not through CloudProxy. - **Quiet since September 2025**: latest release and newest `main` commit share that date. - **Free software, paid pool**: cost scales with instances, bandwidth, and churn. - **Rotation has limits**: Cloudflare can block or challenge by IP or ASN. - **Narrow fit**: static, lightly protected targets. JavaScript needs a browser. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## What Is CloudProxy and What Does It Actually Do? CloudProxy is a free, MIT-licensed tool that turns cloud provider API tokens into a pool of proxy VMs. It creates the machines, installs proxy software, health-checks them, and holds the pool at a target size. It is not a scraper, not a proxy network, and not a service you buy access to. One disambiguation first. This article covers [claffin/cloudproxy](https://github.com/claffin/cloudproxy), which has no relation to Google Cloud's SQL Auth Proxy. The two compete for similar search terms and solve different problems. The repository documents DigitalOcean, AWS, Google Cloud, Hetzner, and Vultr as supported providers. Azure and Scaleway are listed as planned. That list is current as of July 2026. What CloudProxy hands you is provisioning. What you still own is everything around it: cloud credentials, firewall rules, region choices, quotas, instance cleanup, and the bill. Proxy type matters here too. Every address CloudProxy creates belongs to a hosting provider, which puts the whole pool in the datacenter category rather than the residential one. ### Why the CloudProxy API Is Not a Proxy Gateway The local API is a control plane, not a proxy endpoint. `GET /random` returns the connection details of one healthy proxy, and your scraper then opens a connection straight to that VM. The CloudProxy process never sits in the request path and never sees your traffic: bash```bash curl -X 'GET' 'http://localhost:8000/random' -H 'accept: application/json' ``` The response carries the proxy address plus a ready-built connection URL: json```json { "metadata": { "request_id": "123e4567-e89b-12d3-a456-426614174000", "timestamp": "2026-07-28T08:00:00Z" }, "message": "Random proxy retrieved successfully", "proxy": { "ip": "203.0.113.10", "port": 8899, "auth_enabled": true, "url": "http://changeme:changeme123@203.0.113.10:8899", "provider": "digitalocean", "instance": "default" } } ``` That `url` field is the one you pass to your HTTP client. CloudProxy builds it server-side, so you never assemble the credential string yourself. This distinction changes how you debug. Latency, throughput, and blocking behavior all belong to the chosen VM, not to CloudProxy. When a request gets blocked, CloudProxy has nothing to retry, because it never saw the request. With the architecture straight, the next question is whether the project is active enough to depend on. ## Is CloudProxy Still Maintained in 2026? It works, and it is quiet. Both the newest tagged release and the newest commit on the default branch carry the same date. Verified against the GitHub API on 28 July 2026: - Latest tagged release: `v0.6.44`, published 9 September 2025. - Most recent commit on `main`: `06f8a9d`, also dated 9 September 2025. - Open issues: 39. Stars: roughly 1,720. License: MIT. One detail makes this easy to misread. The repository's `pushed_at` timestamp looks recent because dependency-bump branches still get pushed. The default branch has not moved since September 2025. Practically, this means no new provider support is likely soon. Cloud provider API changes and base-image deprecations will need patching on your side. If you depend on this in production, plan to fork it. That is the dependency picture. Here is how you get a pool running. ## How Do You Set Up CloudProxy With Docker? One `docker run` command with a single provider token gets a pool running. Compose earns its place once you want several providers or configuration you can keep in version control. You need two things. Docker installed, and at least one cloud provider API token with permission to create instances. The [Docker install docs](https://docs.docker.com/engine/install/) cover every platform. ### Running CloudProxy With a Single Docker Command This starts CloudProxy with DigitalOcean as the only provider: bash```bash docker run -d \ -e PROXY_USERNAME='changeme' \ -e PROXY_PASSWORD='changeme123' \ -e DIGITALOCEAN_ENABLED=True \ -e DIGITALOCEAN_ACCESS_TOKEN='dop_v1_your_token' \ -p 8000:8000 \ laffin/cloudproxy:latest ``` The variables are `PROXY_USERNAME` and `PROXY_PASSWORD`, not `USERNAME` and `PASSWORD`. Older tutorials get this wrong, and the container starts quietly without the auth you expected. Two URLs are worth opening right away. The scaling UI lives at `localhost:8000/ui`, and interactive API docs live at `localhost:8000/docs`. The docs page is the faster way to explore endpoints. One warning about those credentials. Whatever you pass becomes the proxy authentication for every VM in the pool. Keep them alphanumeric. CloudProxy drops them into the connection URL without encoding, so a `@` or `:` in a password produces a URL that won't parse. ### Configuring DigitalOcean, AWS, Google Cloud, Hetzner, and Vultr For more than one provider, Compose keeps the configuration readable: yaml```yaml services: cloudproxy: image: laffin/cloudproxy:latest restart: unless-stopped ports: - "8000:8000" environment: - PROXY_USERNAME=changeme - PROXY_PASSWORD=changeme123 - ONLY_HOST_IP=True - AGE_LIMIT=0 - DIGITALOCEAN_ENABLED=True - DIGITALOCEAN_ACCESS_TOKEN=dop_v1_your_token - DIGITALOCEAN_MIN_SCALING=3 - DIGITALOCEAN_REGION=lon1 - HETZNER_ENABLED=True - HETZNER_ACCESS_TOKEN=your_token - HETZNER_MIN_SCALING=2 - VULTR_ENABLED=True - VULTR_ACCESS_TOKEN=your_token - VULTR_MIN_SCALING=2 ``` Every entry under `environment` needs the `-` list prefix. Drop it on one line and the whole file fails to parse. There is also no `version` key, which current Compose treats as obsolete. `ONLY_HOST_IP=True` restricts proxy access to the IP of the machine running CloudProxy. Without it, anyone who learns a VM address and your credentials can use your pool. Each provider takes its own region and size variables. You can also run several accounts per provider by adding a name segment to the variable. The repository's troubleshooting notes flag the usual traps. AWS AMI IDs are valid in only one region, and some Vultr plans are missing from certain locations. ### Setting Pool Size With MIN\_SCALING and the Providers Endpoint You can resize a pool without restarting the container. Send a PATCH to the provider endpoint: bash```bash curl -X 'PATCH' 'http://localhost:8000/providers/digitalocean' \ -H 'Content-Type: application/json' \ -d '{"min_scaling": 5, "max_scaling": 5}' curl -X 'GET' 'http://localhost:8000/providers' -H 'accept: application/json' ``` The PATCH sets the target and the GET confirms what exists. Those two numbers differ for a while after any change. The reason is the reconciliation loop. In commit `06f8a9d`, CloudProxy schedules a per-provider manager job every 20 seconds. Each run creates or destroys instances to match the target, so new addresses show up on a delay. `AGE_LIMIT` drives cost more directly than any other setting. Measured in seconds, it forces CloudProxy to destroy and rebuild instances once they pass that age. It ships disabled. Turning it on is what converts a fixed monthly bill into a churning one. With a pool running, you can point a scraper at it. ## How Do You Route a Python Scraper Through CloudProxy? Request a proxy from `/random` and pass its `url` field to your HTTP client. Then handle the case where that proxy is already gone, which is the part most tutorials skip. Two packages cover it. We'll use [httpx](https://pypi.org/project/httpx/) for requests and [parsel](https://pypi.org/project/parsel/) for parsing the HTML. Install both with pip: bash```bash pip install httpx parsel ``` These examples run on httpx 0.28.1 and parsel 1.10.0 under Python 3.14. Pinning matters, because the proxy argument changed shape between httpx releases. ### Pulling a Proxy From the /random Endpoint This scrapes product data from [web-scraping.dev](https://web-scraping.dev/products) through a CloudProxy proxy: python```python import json import httpx from parsel import Selector CLOUDPROXY = "http://localhost:8000" def parse_products(response): """parse products from HTML""" selector = Selector(response.text) data = [] for product in selector.css("div.row.product"): data.append({ "name": product.css("h3 a::text").get(), "link": product.css("h3 a::attr(href)").get(), "price": product.css(".price::text").get(), "image": product.css("img::attr(src)").get(), }) return data def get_proxy(): """ask CloudProxy for one healthy proxy and return its connection URL""" response = httpx.get(f"{CLOUDPROXY}/random", timeout=10) response.raise_for_status() return response.json()["proxy"]["url"] def scrape_products(url): """scrape product data through a CloudProxy proxy""" proxy = get_proxy() response = httpx.get(url, proxy=proxy, timeout=30) response.raise_for_status() return parse_products(response) data = scrape_products("https://web-scraping.dev/products") print(json.dumps(data, indent=2)) ``` Here is the output: json```json [ { "name": "Box of Chocolate Candy", "link": "https://web-scraping.dev/product/1", "price": "24.99", "image": "https://web-scraping.dev/assets/products/orange-chocolate-box-medium-1.webp" }, { "name": "Dark Red Energy Potion", "link": "https://web-scraping.dev/product/2", "price": "4.99", "image": "https://web-scraping.dev/assets/products/darkred-potion.webp" }, { "name": "Teal Energy Potion", "link": "https://web-scraping.dev/product/3", "price": "4.99", "image": "https://web-scraping.dev/assets/products/teal-potion.webp" }, { "name": "Red Energy Potion", "link": "https://web-scraping.dev/product/4", "price": "4.99", "image": "https://web-scraping.dev/assets/products/red-potion.webp" }, { "name": "Blue Energy Potion", "link": "https://web-scraping.dev/product/5", "price": "4.99", "image": "https://web-scraping.dev/assets/products/blue-potion.webp" } ] ``` Two details correct what older guides show. The argument is `proxy`, singular. httpx deprecated `proxies` in 0.26.0 and removed it in 0.28.0. The old form raises a `TypeError` on any current install. The second is the proxy string. It follows `scheme://user:pass@host:port`, with an `@` separating credentials from the host. Guides showing a colon there print something no HTTP client can parse. For patterns beyond picking a random proxy per request, see our guide on [rotating proxies in web scraping](https://scrapfly.io/blog/posts/how-to-rotate-proxies-in-web-scraping). ### Handling Dead Proxies and Retries A proxy that resolved thirty seconds ago may already be gone. The reconciliation loop and `AGE_LIMIT` both destroy instances on their own schedule. Any scraper that grabs one proxy at startup and reuses it will eventually fail. Request a fresh proxy on each attempt: python```python import time def scrape_with_retry(url, attempts=3): """re-request a proxy when one turns out to be dead""" for attempt in range(1, attempts + 1): proxy = get_proxy() try: response = httpx.get(url, proxy=proxy, timeout=30) response.raise_for_status() except (httpx.TransportError, httpx.HTTPStatusError) as exc: print(f"attempt {attempt}/{attempts} failed: {exc!r}") time.sleep(2) continue products = parse_products(response) if not products: print(f"attempt {attempt}/{attempts}: 200 but no products") continue return products raise RuntimeError(f"no working proxy after {attempts} attempts") ``` Notice the empty-result check. A dead proxy usually raises a connection error, which is easy to catch. A blocked request often returns a clean 200 carrying a challenge page instead. That silent case is why the wrapper tests parsed results rather than status code alone. If the parser finds nothing, treat it as a failure and try another address. Now for the part that decides whether any of this is worth running. Scrapfly #### Scale your web scraping effortlessly Scrapfly handles proxies, browsers, and anti-bot bypass — so you can focus on data. [Try Free →](https://scrapfly.io/register)## What Does Running a CloudProxy Pool Actually Cost? CloudProxy is free. A usable pool is not. Cost scales with instance count first, outbound bandwidth second, and replacement churn third. The figures below come from DigitalOcean's published pricing on 28 July 2026. Using one provider keeps the arithmetic checkable. Other providers shift the numbers but not the shape. Price your own region before committing, since regional gaps often exceed the gap between instance tiers. | Input | Value | |---|---| | Entry droplet | 1 vCPU, 512 MiB RAM, 10 GiB SSD | | Price per instance | $4 per month | | Included outbound transfer | 500 GiB per instance | | Overage rate | $0.01 per GiB | | Region assumed | London (`lon1`) | So a five-instance pool costs $20 per month in compute, with 2,500 GiB of outbound transfer included. DigitalOcean pools that allowance across your team rather than per droplet. One busy instance can draw on quota the others left unused. Whether bandwidth costs you anything depends on what you scrape: - **Text-heavy pages at roughly 150 KiB each**: about 17 million pages before you reach the overage rate. - **Image-heavy pages at roughly 3 MiB each**: about 850,000 pages against the same allowance. - **Pulling 5 TiB in a month**: 2,620 GiB over the pool, adding about $26 and overtaking compute. Scale is close to linear. Twenty instances run about $80 per month with 10,000 GiB included. Five instances is a small pool. For real address variety on a demanding target, you'll need considerably more before rotation means much. [How to Stop Wasting Money on ProxiesLearn actionable techniques to slash proxy spending in web scraping projects without sacrificing reliability or scale.](https://scrapfly.io/blog/posts/how-to-stop-wasting-money-on-proxies) ### Instance Count, Bandwidth, and Replacement Churn Readers usually estimate the first driver correctly, underestimate the second, and forget the third. Bandwidth counts every byte leaving the VM. Pages, images, fonts, and tracking scripts all travel back through the proxy as billed egress, which is why image-heavy targets change the arithmetic sharply. Churn surprises people. With `AGE_LIMIT` set, CloudProxy destroys and rebuilds instances continuously. Some providers bill a minimum duration per instance, however briefly it lives. Where that applies, aggressive recycling costs disproportionately more, so check your provider's billing granularity first. ### The Process-Stop Billing Risk CloudProxy runs reconciliation and deletion checks inside its own process. If that process stops, cleanup stops with it. Your provider instances keep existing and keep billing. The mechanism is visible in commit `06f8a9d`. Provider managers run as APScheduler jobs on a 20-second interval, and that commit defines no shutdown hook to destroy instances on exit. Keep the boundary precise. Restarting CloudProxy with the same configuration lets reconciliation resume and pick the instances back up. The tool does not permanently lose track of them. But while it is down, nothing is watching. Three habits cover the risk. Set a billing alert, keep identifiable instance names where the provider supports them, and check the console after any long outage. Cost is one constraint. Address reputation is the other, and it does not respond to spending more. ## Why Can Cloud Datacenter IPs Be Blocked Before the Page Loads? A site can apply a rule to the visitor's IP address or Autonomous System Number. Cloudflare's IP Access rules officially support allowlist, block, and challenge actions. Those actions match on IP address, IP range, ASN, or country, per its [WAF documentation](https://developers.cloudflare.com/waf/tools/ip-access-rules/). An autonomous system is one network under a single operator's control, identified by a number. Cloud provider ranges map to well-known ASNs, so a rule can match the network identity directly. That is what matters for a proxy pool. Rotating between instances changes the address but usually not the ASN. When a rule targets the network, rotating inside it changes nothing. That is why more IPs from the same provider often fails to help. The limits of this claim matter. Not every anti-bot vendor keeps the same lists, and none of this says ASN checks run first. It is one documented mechanism, not an explanation for every 403. The symptoms are easier to act on than the theory: - **Immediate 403 with no challenge page**: an IP or ASN rule is one possible cause, but so are TLS fingerprint and header mismatches. - **A 200 response containing a challenge**: often misread as a parsing bug, because the request succeeded and the parser found nothing. - **Content that appears only after JavaScript runs**: CloudProxy forwards HTTP and runs no browser, so you get the initial shell. - **Geographic mismatch**: cloud regions are coarse. An address in the right country still reads as hosting, not a home connection. You can watch this play out on a protected target. Point a CloudProxy pool at a site like the one in our [leboncoin scraping guide](https://scrapfly.io/blog/posts/how-to-scrape-leboncoin-marketplace-real-estate). The response is a [DataDome](https://scrapfly.io/blog/posts/how-to-bypass-datadome-anti-scraping) challenge rather than listings. Swapping to another instance in the same provider range returns that same challenge. Our guide on [avoiding IP address blocking](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-ip-addresses) covers the broader set of signals. None of this makes datacenter addresses useless. Plenty of targets never check. The point is narrower: direct network rules are one failure mode a bigger pool cannot solve. ## When Is CloudProxy Still the Right Tool? CloudProxy fits three conditions together. Your targets are lightly protected, your volume needs only a handful of instances, and you value control over your time. The practical framing is to tier your proxies by target difficulty. Many scraping targets never inspect much beyond the address. Serving those from a datacenter pool costs roughly an order of magnitude less than routing everything through residential addresses. Good fits are concrete. They include public documentation, static marketing pages, RSS feeds, small forums, internal endpoints, and any target you own. Poor fits are equally concrete. Targets where testing shows your cloud range is blocked, anything that renders only after JavaScript, and workloads needing consumer-network trust. Add any job where one blocked run costs more than the hosting you saved. | Approach | Best-fit workload | What you operate | Where it fails | Cost shape | |---|---|---|---|---| | **Self-hosted datacenter pool** | Static, lightly protected targets at modest volume | Cloud accounts, tokens, firewall rules, cleanup, the bill | Network-level rules, JavaScript, anti-bot challenges | Fixed monthly compute plus metered egress | | **Managed scraping API** | Protected targets needing rendering and bypass in one request | Nothing beyond your own code | Workloads needing full interactive browser control | Per-request, scales with volume | | **Managed browser** | Interactive flows, logins, multi-step navigation, agent runs | Your automation script only | Simple bulk fetches, where it is more machinery than needed | Billed by active browser time | One note on the maintenance question. Depending on a quiet project is a defensible call when the job it does is simple and stable, and provisioning VMs largely is. If your workload lands in the first row, run CloudProxy. If it doesn't, the reason is almost always one of two things CloudProxy cannot do at all: render JavaScript or get past anti-bot detection, cover ed next. CloudProxy scraping blocking## What CloudProxy Cannot Do: JavaScript, Anti-Bot, and Browser Fingerprints CloudProxy forwards HTTP requests. It does not run a browser, execute JavaScript, manage fingerprints, or solve challenges. These sit outside what the tool does. The consequences compound. On a client-rendered target, a proxy-only HTTP client returns an empty shell. On a target with a matching network rule, the request fails before rendering would matter. A correctly configured pool can still hand you unusable content. The missing layer is a real browser with a coherent fingerprint, reached from an address the target trusts. That is a different category of machinery than a proxy pool. ScrapFly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) is a single HTTP endpoint for collecting web data at scale, with a **99.99% success rate** across **130M+ proxies in 190+ countries**. - [Anti-Scraping Protection bypass](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - automatically defeats Cloudflare, DataDome, PerimeterX, Akamai, and 90+ other bot systems. - [Smart proxy rotation](https://scrapfly.io/docs/scrape-api/proxy) - residential and datacenter pools with country and ASN level geo-targeting. - [JavaScript rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - render SPAs and dynamic pages through real cloud browsers. - [Browser automation scenarios](https://scrapfly.io/docs/scrape-api/javascript-scenario) - scroll, click, fill forms, and wait for elements without managing a browser fleet. - [Format conversion](https://scrapfly.io/docs/scrape-api/getting-started#api_param_format) - return pages as HTML, JSON, clean text, or LLM ready Markdown. - [Session management](https://scrapfly.io/docs/scrape-api/session) - keep cookies, headers, and IPs consistent across multi step flows. - [Smart caching](https://scrapfly.io/docs/scrape-api/getting-started#api_param_cache) - cache successful responses to cut cost on repeat scraping jobs. - [Python](https://scrapfly.io/docs/sdk/python), [TypeScript](https://scrapfly.io/docs/sdk/typescript), [Scrapy](https://scrapfly.io/docs/sdk/scrapy), and [no-code integrations](https://scrapfly.io/docs/integration/getting-started) including [Make](https://scrapfly.io/integration/make), [n8n](https://scrapfly.io/integration/n8n), [Zapier](https://scrapfly.io/integration/zapier), [LangChain](https://scrapfly.io/integration/langchain), and [LlamaIndex](https://scrapfly.io/integration/llamaindex). Here is the same scrape through that endpoint, including the pool you no longer run: python```python from scrapfly import ScrapflyClient, ScrapeConfig client = ScrapflyClient(key="YOUR_SCRAPFLY_KEY") result = client.scrape(ScrapeConfig( url="https://web-scraping.dev/products", # render client-side content render_js=True, # handle anti-bot protection asp=True, country="us", )) print(result.scrape_result["content"]) ``` The brevity is the argument. There is no pool to size, no reconciliation delay, and no orphaned instance to clean up after a crash. ### Running AI Browser Agents on Managed Cloud Browsers If you already run your own servers, the browser layer is the same tradeoff one level up. The operational burden is much higher, because browsers need more patching and more memory than proxy VMs. Cloud Browser serves stealth Chromium over a CDP WebSocket. Existing Playwright, Puppeteer, and Selenium code connects by swapping the endpoint URL, which is standard CDP compatibility. [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use) connects the same way, by pointing its `cdp_url` at the Scrapfly WebSocket endpoint: python```python import asyncio from browser_use import Agent, Browser, BrowserProfile from langchain_openai import ChatOpenAI API_KEY = "YOUR_SCRAPFLY_KEY" BROWSER_WS = f"wss://browser.scrapfly.io?api_key={API_KEY}&proxy_pool=public_datacenter_pool&os=linux" async def run_agent(): browser = Browser(browser_profile=BrowserProfile(cdp_url=BROWSER_WS)) agent = Agent( task="Go to https://web-scraping.dev and find the product prices", llm=ChatOpenAI(model="gpt-4o"), browser=browser, ) print("Agent result:", await agent.run()) asyncio.run(run_agent()) ``` The proxy pool, operating system fingerprint, and country are query parameters on that WebSocket URL. Sessions bill for active browser time, so close them when the run finishes. ### Web Scraping API Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in. [Try Web Scraping API](https://scrapfly.io/docs/scrape-api/getting-started) ## FAQ Is CloudProxy free?The software is free under the MIT license, but the hosting is not. You pay your cloud provider for every instance it creates plus outbound bandwidth. Which cloud providers does CloudProxy support?CloudProxy supports DigitalOcean, AWS, Google Cloud, Hetzner, and Vultr, with Azure and Scaleway planned. Multiple accounts per provider work too, which widens your address range. Can CloudProxy bypass Cloudflare or DataDome?Not reliably. It changes the proxy address but adds no browser execution or anti-bot handling, and protected sites can block or challenge by IP and ASN. Does CloudProxy render JavaScript?No. It forwards HTTP requests and never runs a browser, so client-side content never appears in the response. What is the difference between a datacenter and a residential proxy?A datacenter address belongs to a hosting provider, while a residential address belongs to a consumer internet connection. Anti-bot vendors treat the underlying network as a trust signal. Is CloudProxy the same as Google Cloud SQL Auth Proxy?No, though the two are often confused in search. CloudProxy is an open-source scraping proxy pool manager, while Cloud SQL Auth Proxy authenticates database connections. ## Summary CloudProxy solves the provisioning problem well and does not solve the trust problem at all. It turns cloud API tokens into a working, health-checked proxy pool with little effort on your side. The code to use it is short, once you know the API is a control plane rather than a gateway. The cost is honest but not zero. A five-instance DigitalOcean pool runs about $20 a month before bandwidth and scales close to linearly. It also needs cleanup discipline, because instances keep billing when CloudProxy is not running. Where it stops working is worth stating plainly. If your targets check little beyond the address, a small self-hosted pool is genuinely cheap and worth running. If they check anything more, no pool size fixes it. The rejection happens at the network level, before your request is ever evaluated. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents- [Key Takeaways](#key-takeaways) - [What Is CloudProxy and What Does It Actually Do?](#what-is-cloudproxy-and-what-does-it-actually-do) - [Why the CloudProxy API Is Not a Proxy Gateway](#why-the-cloudproxy-api-is-not-a-proxy-gateway) - [Is CloudProxy Still Maintained in 2026?](#is-cloudproxy-still-maintained-in-2026) - [How Do You Set Up CloudProxy With Docker?](#how-do-you-set-up-cloudproxy-with-docker) - [Running CloudProxy With a Single Docker Command](#running-cloudproxy-with-a-single-docker-command) - [Configuring DigitalOcean, AWS, Google Cloud, Hetzner, and Vultr](#configuring-digitalocean-aws-google-cloud-hetzner-and-vultr) - [Setting Pool Size With MIN\_SCALING and the Providers Endpoint](#setting-pool-size-with-min-scaling-and-the-providers-endpoint) - [How Do You Route a Python Scraper Through CloudProxy?](#how-do-you-route-a-python-scraper-through-cloudproxy) - [Pulling a Proxy From the /random Endpoint](#pulling-a-proxy-from-the-random-endpoint) - [Handling Dead Proxies and Retries](#handling-dead-proxies-and-retries) - [What Does Running a CloudProxy Pool Actually Cost?](#what-does-running-a-cloudproxy-pool-actually-cost) - [Instance Count, Bandwidth, and Replacement Churn](#instance-count-bandwidth-and-replacement-churn) - [The Process-Stop Billing Risk](#the-process-stop-billing-risk) - [Why Can Cloud Datacenter IPs Be Blocked Before the Page Loads?](#why-can-cloud-datacenter-ips-be-blocked-before-the-page-loads) - [When Is CloudProxy Still the Right Tool?](#when-is-cloudproxy-still-the-right-tool) - [What CloudProxy Cannot Do: JavaScript, Anti-Bot, and Browser Fingerprints](#what-cloudproxy-cannot-do-javascript-anti-bot-and-browser-fingerprints) - [Running AI Browser Agents on Managed Cloud Browsers](#running-ai-browser-agents-on-managed-cloud-browsers) - [FAQ](#faq) - [Summary](#summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ blocking proxies ### The Complete Guide To Using Proxies For Web Scraping Introduction to proxy usage in web scraping. What types of proxies are there? How to evaluate proxy providers and avoid ... ](https://scrapfly.io/blog/posts/introduction-to-proxies-in-web-scraping) [ proxies ### The Best Datacenter Proxies in 2026: A Complete Guide Datacenter proxies are a top choice for web scraping, automation, and online anonymity thanks to their speed, low cost, ... ](https://scrapfly.io/blog/posts/the-best-datacenter-proxies) [ headless-browser scaling ### Web Scraping With Cloud Browsers Introduction cloud browsers and their benefits and a step-by-step setup with self-hosted Selenium-grid cloud browsers. ](https://scrapfly.io/blog/posts/web-scraping-with-cloud-browsers) ## Related Questions - [ Q Python httpx vs requests vs aiohttp - key differences ](https://scrapfly.io/blog/answers/httpx-vs-requests-vs-aiohttp) Premium rotating proxies for scraping, **1,000 free credits** [Start Free](https://scrapfly.io/register)