[Blog](https://scrapfly.io/blog) / [proxies](https://scrapfly.io/blog/tag/proxies) / [4 Best Open-Source Proxy Scrapers and Checkers in 2026](https://scrapfly.io/blog/posts/best-proxy-scrapers) # 4 Best Open-Source Proxy Scrapers and Checkers in 2026 by [Ziad Shamndy](https://scrapfly.io/blog/author/ziad) Oct 01, 2026 15 min read [\#proxies](https://scrapfly.io/blog/tag/proxies) [\#tools](https://scrapfly.io/blog/tag/tools) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers&text=4%20Best%20Open-Source%20Proxy%20Scrapers%20and%20Checkers%20in%202026 "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-proxy-scrapers) **Proxy Saver**Optimize your existing proxies with bandwidth savings and fingerprint fortification. [ Learn More ](https://scrapfly.io/products/proxy-saver) [ Docs ](https://scrapfly.io/docs/proxy-saver/getting-started) Downloading a few thousand public proxy addresses feels like progress right up until the checker runs. Most of that list turns out to be dead sockets, half open connections, and addresses that answer a ping but refuse the actual request. This guide covers four current open source tools and the exact job each one does. The guide also maps the shortest route from a raw address list to a result validated against a real target. A proxy scraper collects candidates. A proxy checker proves which ones work. A data feed skips straight to candidates someone else already collected. A 2024 longitudinal study tracked more than 640,600 free proxies from 11 providers over 30 months and found that 34.5% were active at least once. That figure describes historical activity across the whole study, not a reusable success rate, and it says nothing about whether a single proxy works against your target today. Every repository fact, release date, and license below reflects a live check performed on 2026-09-21. Recheck the repositories before depending on any of these numbers. ## Key Takeaways A quick summary before the full breakdown: - A proxy scraper collects candidate addresses. A proxy checker proves which candidates can complete a request. For most workflows, checking is the more important step. - Use a documented TXT or JSON feed when one exists. Scrape or extract an HTML table only when the source offers no machine readable output. - An extraction model can normalize irregular HTML, but it cannot validate liveness, latency, anonymity, or target compatibility. Those checks require real network requests. - The 2024 study above tracked more than 640,600 free proxies for 30 months and found 34.5% active at least once. One successful check still does not establish production reliability. - Public proxies are untrusted infrastructure. Never send credentials, cookies, tokens, personal data, or sensitive payloads through one. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## Which Open Source Proxy Scraper Is Best? Use monosans/proxy-scraper-checker for a single scrape and check run, jhao104/proxy\_pool for a persistent JSON API, ProxyBroker2 for an async finder with a local proxy server, and mubeng when a candidate list already exists and only needs checking and rotation. The table below narrows the choice to one line per job: | Tool | Best for | Primary output | Needs input list | |---|---|---|---| | monosans/proxy-scraper-checker | Scrape + check run | JSON + TXT | No | | jhao104/proxy\_pool | Persistent pool | JSON API | No | | ProxyBroker2 | Find + serve | CLI + local proxy | No | | mubeng | Check + rotate | TXT + local proxy | Yes | Repository health tells a separate story from the job table above, so check it on its own axis: | Tool | Language | License | Default-branch activity | Latest release | |---|---|---|---|---| | monosans/proxy-scraper-checker | Rust | MIT | 2026-09-21 | Nightly builds | | jhao104/proxy\_pool | Python | MIT | 2026-06-15 | 2.4.1, 2023-02-23 | | ProxyBroker2 | Python | Apache-2.0 | 2026-06-09 | 2.0.0b3, 2026-05-09 | | mubeng | Go | Apache-2.0 | 2026-09-09, docs | 0.23.0, 2025-08-02 | Both tables come from the same 2026-09-21 snapshot of each repository's default branch and release history. A recent commit and a recent tagged release are different signals. The mubeng row above shows exactly why, since its newest commit only touched documentation while its last binary release is more than a year old. ## Do You Need a Proxy Scraper, a Proxy Checker, or a Free Proxy Feed? A feed hands you candidate data someone already collected. A scraper builds that candidate list from scratch. A checker sends real requests to prove which candidates work. These are three different jobs, not interchangeable products. The starting point decides the route: | Starting point | Use | Output | Next step | |---|---|---|---| | TXT or JSON feed | Parse directly | Candidate proxies | Validate | | Multiple public sources | Scraper + checker | Filtered list | Target-check | | Irregular HTML only | Parser or extraction model | Structured entries | Validate | | Existing list | Checker or rotator | Live subset / local proxy | Monitor | mermaid```mermaid flowchart LR A[Feed or HTML Source] --> B[Parse or Extract] B --> C[Network Check] C --> D[Target-Specific Valid Output] ``` The diagram separates extraction from validation on purpose. Parsing a page or a feed only produces structured candidates, and only the network check step proves any of them still work. A documented feed beats scraping its own HTML presentation whenever one exists. Fewer moving parts, explicit fields, and no selector to maintain when the page layout changes. [monosans/proxy-list](https://github.com/monosans/proxy-list) is a concrete example of that kind of feed. The repository republishes HTTP, SOCKS4, and SOCKS5 lists as plain text plus a JSON file. The JSON file carries response time, exit IP, ASN, and city level geolocation for each entry. A checker covered later in this guide refreshes the feed on a schedule. bash```bash curl -fsSL https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/socks5.txt ``` The command pulls the current SOCKS5 list directly from GitHub as raw text, with no signup and no API key. Parsing that file is enough to get candidates. Every line still needs a real network check before it counts as usable. A repository of generated proxy files is a data feed, not another ranked scraper or checker tool. Keeping that distinction exact avoids treating a static export as if it had the same maintenance guarantees as the software that built it. ### Can AI Extract and Validate a Free Proxy List? An extraction model can turn an inconsistent HTML table into structured `protocol`, `host`, and `port` fields. It cannot validate a proxy, because liveness, latency, exit IP, and target compatibility all require an actual network request. Treat AI extraction as conditional rather than default. When a source already exposes TXT, JSON, CSV, or an API, parse that structured output directly instead of extracting from the rendered page. When HTML is genuinely the only source, validate the model's output against a schema before trusting it: - The `protocol` field matches a known enum such as `http`, `socks4`, or `socks5`. - The `host` field is a valid IP address or hostname, and `port` is an integer from 1 through 65535. - Duplicate entries are removed by the combination of protocol, host, and port. After parsing, send a real HTTPS request through each candidate and confirm the response status and content match a controlled target. A candidate that fails that check is not a working proxy no matter how clean the extracted row looks. ## 1. monosans/proxy-scraper-checker: Best for One Scrape and Check Run [monosans/proxy-scraper-checker](https://github.com/monosans/proxy-scraper-checker) is the best default when one binary should collect HTTP, SOCKS4, and SOCKS5 candidates, fetch a real URL through each one, deduplicate the results, and write JSON plus text files in a single run. The tool is written in Rust and licensed under MIT. Its default branch was active as recently as 2026-09-21 in this snapshot. The project ships no tagged GitHub releases and instead distributes nightly binary builds from every run of its CI pipeline. Output is a fixed contract rather than a loose convention. A run writes compact and pretty printed JSON plus `all.txt` with a protocol prefix, alongside separate text files per protocol. The `check_url` setting decides how much metadata a surviving proxy carries. The documented default, `https://ipv4.icanhazip.com`, returns only the caller's IP address, so the tool can populate exit IP, ASN, and geolocation fields automatically. Pointing `check_url` at an ordinary page still proves a proxy can complete a full request. That choice leaves exit IP, ASN, and geolocation empty instead. The repository's own README carries a safety warning worth repeating. Checking opens hundreds of simultaneous connections to untrusted hosts, which can read as abusive traffic to an ISP or overwhelm a cheap router's connection table. toml```toml # config.toml (minimal keys to start) [output] path = "./out" [output.txt] enabled = true [output.json] enabled = true [checking] check_url = "https://ipv4.icanhazip.com" [scraping.http] enabled = true urls = [ "https://api.proxyscrape.com/v2/?request=displayproxies&protocol=http", ] ``` bash```bash git clone https://github.com/monosans/proxy-scraper-checker.git cd proxy-scraper-checker cargo run --features tui --release --locked ``` The config enables both text and JSON output, points `check_url` at the IP echo endpoint, and lists one real HTTP source to scrape. The `cargo run` command builds the release binary and writes validated results into `./out`. The honest limit is source quality rather than tool quality. The tool proves a candidate completed one configured check at one point in time. That proof cannot guarantee the same address still works an hour later or against a different target. A single run like this is ideal for a list built once and used immediately. The next tool trades that one-shot model for a service that keeps running and answers over an API. ## 2. jhao104/proxy\_pool: Best for a Persistent JSON Proxy Pool [jhao104/proxy\_pool](https://github.com/jhao104/proxy_pool) is the best fit when a scheduled fetch and check service should expose a continuously updated pool through HTTP endpoints, instead of producing a one-time file. The project is written in Python under the MIT license. Its default-branch commit is dated 2026-06-15, and its latest tagged release, `2.4.1`, dates back to 2023-02-23. Redis stores the pool. A scheduler process fetches and checks candidates on an interval, and a separate API process serves or removes entries. The current README documents five routes worth knowing: `/get` returns one proxy and accepts a `type=https` filter, `/pop` returns and removes one, `/all` lists the whole pool, `/count` reports pool size, and `/delete` removes a specific address. Two caveats are worth flagging up front. Project documentation is primarily written in Chinese, and the README also carries third party sponsor content that this guide does not repeat or link. yaml```yaml # docker-compose.yml services: redis: image: redis:7 proxy_pool: image: jhao104/proxy_pool:latest depends_on: - redis environment: DB_CONN: redis://redis:6379/0 ports: - "5010:5010" ``` python```python import requests proxy = requests.get( "http://127.0.0.1:5010/get?type=https", timeout=10, ).json()["proxy"] response = requests.get( "https://web-scraping.dev/products", proxies={"https": f"http://{proxy}"}, timeout=15, ) print(response.status_code, len(response.text)) ``` The Compose file runs Redis alongside the pool container and exposes the API on port 5010, matching the connection string in `DB_CONN`. The [requests](https://requests.readthedocs.io/) script pulls one HTTPS capable proxy from `/get` and routes a real request through it to web-scraping.dev/products. The honest limit is that a pool API centralizes refresh and eviction logic, but the free proxies underneath it stay exactly as untrusted and volatile as they would be anywhere else. A pool answers over an API, but it does not act as a proxy server itself. The next tool finds, checks, and serves proxies from a single async Python package instead. ## 3. ProxyBroker2: Best for Async Finding and a Local Proxy Server [ProxyBroker2](https://github.com/bluet/proxybroker2) is the best pick when Python should find, check, filter, and serve public proxies through a local rotating endpoint in one package. The project is licensed under Apache-2.0, with a default-branch commit dated 2026-06-09 in this snapshot. The current stable release is the prerelease tag `v2.0.0b3`, published 2026-05-09, and it is worth treating that beta label as accurate rather than a formality. Supported Python versions run from 3.10 through 3.14. The CLI exposes four commands: `find` searches and validates proxies, `grab` collects without checking, `serve` runs a local rotating proxy server, and `update-geo` downloads a detailed GeoIP database. The original `proxybroker` package on PyPI is version 0.3.2 and no longer maintained, so install the maintained fork directly from its tagged GitHub release rather than from PyPI. bash```bash pip install -U "git+https://github.com/bluet/proxybroker2.git@v2.0.0b3" python -m proxybroker find --types HTTP HTTPS --lvl High --limit 10 python -m proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS ``` bash```bash curl -x http://127.0.0.1:8888 https://web-scraping.dev/products ``` The `pip install` command pins the maintained fork to its current beta tag rather than the stale PyPI package. The `find` command prints ten high anonymity HTTP and HTTPS proxies. The `serve` command turns the same finder into a rotating local proxy. The curl request routes through that local proxy on its way to web-scraping.dev/products. [How to Build a Proxy Rotation API With mitmproxyBuild a rotating proxy API with mitmproxy. Add sticky sessions, health scoring, client-owned retries, and safe caching, then compare managed and BYOP routes.](https://scrapfly.io/blog/posts/build-a-proxy-api-rotate-proxies-and-save-bandwidth) Beta status and a large inherited codebase make ProxyBroker2 a deliberate choice rather than a drop-in replacement for managed infrastructure. The next tool skips finding proxies entirely and focuses only on checking and rotating a list you already have. Scrapfly #### Scale your web scraping effortlessly Scrapfly handles proxies, browsers, and anti-bot bypass — so you can focus on data. [Try Free →](https://scrapfly.io/register)## 4. mubeng: Best for Checking and Rotating an Existing Proxy List [mubeng](https://github.com/mubeng/mubeng) is the best choice when a candidate file already exists and one Go binary should check it, write the live subset, and expose that subset through a local rotating HTTP proxy. The project is written in Go under the Apache-2.0 license, with a latest tagged release of `v0.23.0` from 2025-08-02. Its default branch moved as recently as 2026-09-09, but that commit only reverted a documentation change. Treat the release date as the real measure of the last product update. Supported input schemes cover HTTP, HTTPS, SOCKS4, and SOCKS5, and the local rotator itself serves plain HTTP. bash```bash mubeng -f proxies.txt --check --output live.txt mubeng -f live.txt --address 127.0.0.1:8080 --rotate 1 ``` The first command checks every entry in `proxies.txt` and writes only the surviving addresses to `live.txt`. The second command serves that validated list as a local rotating proxy on port 8080, switching to the next address on every request. The honest limit is scope. mubeng does not harvest candidates on its own, so pair it with a feed or a scraper and describe it as a checker and rotator rather than a proxy scraper. ## How Do You Check Whether a Proxy Scraper Is Still Maintained? Check the default branch, the tagged releases, and the license as three separate signals. A recent documentation commit is not the same thing as a current binary release. The mubeng row in the earlier table is a direct example of that gap. Three checks decide whether a repository is worth adopting: - The default-branch commit date, and what that commit actually changed. - The latest release date, and whether the tag is marked stable or a prerelease. - A current build or test result for the runtime versions the project claims to support. License is a separate adoption gate from activity. A public repository with no license attached is visible source code, not reusable open source software, regardless of how active its commit history looks. ProxyBroker2 illustrates why all three checks matter together. Its default branch moved in June 2026, yet its newest installable tag is still an explicitly labeled beta, so a fresh commit history does not by itself mean a stable release exists. ## Why Do Free Public Proxies Fail in Production? A public proxy list is a queue of untrusted candidates, not available capacity. Availability, traffic integrity, and compatibility with a specific target all fail independently of each other. Two academic studies measured this directly, and their numbers describe different windows and should not be averaged together: - A [2018 study](https://arxiv.org/abs/1806.10258) tested more than 107,000 listed open proxies across 13 million requests over 50 days and found more than 92% unresponsive to proxy requests. - A [2024 study](https://arxiv.org/abs/2403.02445) tracked more than 640,600 free proxies from 11 providers over 30 months, found 34.5% active at least once, and identified 16,923 proxies that manipulated the content passing through them. Four failure modes explain most production breakage: - **Dead or slow candidates** consume worker capacity on connection timeouts before a scraper ever reaches the target. - **Target-specific blocks** happen even after a proxy passes a generic liveness check, because passing an IP echo check says nothing about a specific site's defenses. - **Traffic integrity risk** is real and measured. The 2024 study above found content manipulation happening on thousands of collected proxies whose operators are unknown. - **Silent bad output** occurs when a `200` status code still hides a challenge page or altered content, so a scraper needs to check for an expected marker, not just the status code. Current practitioner discussion in web scraping communities makes the same distinction these two studies do. Proving a proxy answered once is not the same as proving it is reliable enough to depend on. A [recent discussion thread](https://www.reddit.com/r/WebScrapingInsider/comments/1sk6jo4/free_proxy_lists_actually_useful_for_web_scraping/) covers this exact gap between liveness and reliability in more detail. Because the operator behind a free public proxy is unknown, treat it as untrusted infrastructure by default. Never send credentials, cookies, session tokens, personal data, or any other sensitive payload through one. [What Is a Proxy Server?A proxy server is one of those technologies every developer has heard of, but few truly understand beyond the basics of "it hides my IP address." In reality, proxies sit at the heart of modern networking and enable everything from corporate firewalls to the massive data-collection pipelines that...](https://scrapfly.io/blog/posts/what-is-a-proxy-server) When a job needs reliable page responses rather than raw proxy inventory, Scrapfly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) handles proxy routing, retries, rendering, and anti-bot challenge handling inside a single request, so the maintenance loop above stops being the reader's problem. [The Complete Guide To Using Proxies For Web ScrapingIntroduction to proxy usage in web scraping. What types of proxies are there? How to evaluate proxy providers and avoid common issues.](https://scrapfly.io/blog/posts/introduction-to-proxies-in-web-scraping) ## FAQ Is using a public proxy list legal?Legality depends on the jurisdiction, the target, and what you do through the proxy. A publicly listed endpoint does not establish the operator's consent, permission to access a target, or trustworthiness. This is not legal advice, so a specific question deserves a lawyer. Does HTTPS make a free public proxy safe?HTTPS with certificate verification protects the page contents themselves from passive reading or tampering. The proxy operator can still see which hosts you connect to and when, and can drop connections at will. Never disable certificate verification for an unknown public proxy. Are HTTP, HTTPS, SOCKS4, and SOCKS5 proxy entries interchangeable?No. The client has to support the specific protocol a listed entry uses, and an HTTPS target routed through an HTTP proxy normally needs CONNECT tunneling. Preserve the protocol scheme in output files like `all.txt` rather than guessing it from the port number. Do free proxy scrapers still work in 2026?Yes, for learning, testing failure paths, and disposable experiments against public pages. They do not turn a list of public addresses into reliable production infrastructure, and every candidate still needs a fresh, target-specific check before you depend on it. ## Conclusion Pick by job rather than by adjective. monosans/proxy-scraper-checker covers a one-run scrape and check, and jhao104/proxy\_pool covers a persistent API backed pool. ProxyBroker2 covers async finding plus a local proxy server, and mubeng covers checking and rotating a list you already have. The route boundary matters as much as the tool choice. Use a structured feed directly when one exists, and reach for extraction only when the source is irregular HTML. Treat a network check as the only step that actually proves a proxy works. For readers who need reliable page responses instead of raw public proxy maintenance, Scrapfly's Web Scraping API covers proxy selection, rendering, and anti-bot handling behind one request, which removes the list upkeep this guide walks through. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents- [Key Takeaways](#key-takeaways) - [Which Open Source Proxy Scraper Is Best?](#which-open-source-proxy-scraper-is-best) - [Do You Need a Proxy Scraper, a Proxy Checker, or a Free Proxy Feed?](#do-you-need-a-proxy-scraper-a-proxy-checker-or-a-free-proxy-feed) - [Can AI Extract and Validate a Free Proxy List?](#can-ai-extract-and-validate-a-free-proxy-list) - [1. monosans/proxy-scraper-checker: Best for One Scrape and Check Run](#1-monosans-proxy-scraper-checker-best-for-one-scrape-and-check-run) - [2. jhao104/proxy\_pool: Best for a Persistent JSON Proxy Pool](#2-jhao104-proxy-pool-best-for-a-persistent-json-proxy-pool) - [3. ProxyBroker2: Best for Async Finding and a Local Proxy Server](#3-proxybroker2-best-for-async-finding-and-a-local-proxy-server) - [4. mubeng: Best for Checking and Rotating an Existing Proxy List](#4-mubeng-best-for-checking-and-rotating-an-existing-proxy-list) - [How Do You Check Whether a Proxy Scraper Is Still Maintained?](#how-do-you-check-whether-a-proxy-scraper-is-still-maintained) - [Why Do Free Public Proxies Fail in Production?](#why-do-free-public-proxies-fail-in-production) - [FAQ](#faq) - [Conclusion](#conclusion) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ http proxies ### SOCKS5 vs HTTP Proxy: Key Differences and When to Use Each Compare HTTP and SOCKS5 proxies for scraping: CONNECT tunnels, DNS resolution, authentication, UDP/QUIC limits, and test... ](https://scrapfly.io/blog/posts/https-vs-socks-proxies) [ python crawling ### Guide to List Crawling: Everything You Need to Know Complete list crawling tutorial assess site defenses, bypass anti-bot systems, choose tools (Beautiful Soup, Playwright,... ](https://scrapfly.io/blog/posts/guide-to-list-crawling) [ python crawling ### 10 Best Open-Source Web Scrapers in 2026 Ranked by maintenance, license, and production capability. The only neutral open-source scraper list with no entries fro... ](https://scrapfly.io/blog/posts/best-open-source-web-scrapers) ## Related Questions - [ Q What are SOCKS5 proxies and how they compare to HTTP proxies? ](https://scrapfly.io/blog/answers/what-are-socks5-proxies-in-web-scraping) - [ Q How to use proxies with Python httpx? ](https://scrapfly.io/blog/answers/how-to-use-proxies-python-httpx) Premium rotating proxies for scraping, **1,000 free credits** [Start Free](https://scrapfly.io/register)