# The Best Web Scraping CLI ## Agentic CLI for the Scrapfly platform. Run web scraping from your terminal with Scrapfly CLI. Scrape, screenshot, extract, classify, crawl, and drive AI agents from your shell or CI pipeline. Every command returns a stable JSON envelope, pipe-friendly and tool-use ready. [ Get Free API Key ](https://scrapfly.io/register) [ View on GitHub ](https://github.com/scrapfly/scrapfly-cli) 1,000 free credits. No credit card required. scrapfly - terminal scrapfly scrape https://web-scraping.dev/products --format markdown { "success": true, "product": "scrape", "data": "# Products\n\n..." } scrapfly extract --url https://web-scraping.dev/product/1 \ --prompt "product name, price, in_stock" { "success": true, "product": "extract", "data": { "name": "Widget Pro", "price": 49.99 } } scrapfly agent "collect top 10 prices from web-scraping.dev" Planning task ... scraping 10 pages ... extracting ... Done. 10 results written to prices.json --- ## 7 product commands ## 4 LLM providers supported ## JSON stable output envelope ## free open source, MIT licensed --- // FORMULA## Give super powers to your agent. `SKILL.md` + `scrapfly` = all you need No SDK wrapper. No MCP server. No tool-use schema. Hand your agent the skill + the binary, done. --- - Scrape - Screenshot - Extract - Crawler - Browser - Batch - Classify - Agent - MCP Commands ## Every Scrapfly Product. One Command. Each command returns { success, product, data | error }, stable across versions. Scrape Fetch any URL with anti-bot bypass, JS rendering, and proxy rotation built in. Returns clean HTML, markdown, or text. $> `scrapfly scrape --help` Screenshot Capture full-page or element screenshots as PNG, JPEG, or WebP. Returns base64 payload or a CDN URL, ready for downstream agents. $> `scrapfly screenshot --help` Extract Turn HTML into structured data with a prompt or a schema. LLM-powered, schema-validated, deterministic output envelope. $> `scrapfly extract --help` Crawler Traverse entire sites with follow rules, depth limits, and rate control. Streams URLs as they are discovered; every page runs through Scrape. $> `scrapfly crawl --help` Browser Launch a real stealth Chromium session with CDP access. Same envelope as the other commands; attach Playwright or Puppeteer when you need DOM interaction. $> `scrapfly browser --help` Batch Submit a list of URLs in a single call; results stream back as an NDJSON array. Built for bulk pipelines and tool calls. $> `scrapfly batch --help` Classify Tag content into a taxonomy with an LLM pass. Returns label + confidence per record, same envelope, same stability. $> `scrapfly classify --help` Agent Describe an objective in plain language. The agent plans, scrapes, extracts, and returns a structured result. Works with Anthropic, OpenAI, Gemini, or Ollama. $> `scrapfly agent --help` MCP The CLI ships an MCP server. Point Claude Desktop, Cursor, Claude Code, or any MCP-compatible client at the binary and it exposes scrape, extract, screenshot, crawl, agent, and the rest as tool calls automatically. $> `scrapfly mcp serve --help` [Get Free API Key](https://scrapfly.io/register) --- AGENT MODE## Built for Agents Hand your agent the CLI binary and the onboarding skill. It takes it from there. No SDK wrapper, no MCP server, no tool-use schema to wire. ##### Ships With An Agent Skill The CLI repo includes `agent-onboarding/SKILL.md` — a drop-in context file that teaches any LLM what the CLI does, when to use each command, and how to read the JSON envelope. Load it into your system prompt or your agent framework's skill registry and the model is productive immediately. \# Give your agent the skill + the CLI. Done. $ curl -fsSL https://raw.githubusercontent.com/scrapfly/scrapfly-cli/main/agent-onboarding/SKILL.md [Read the skill on GitHub](https://github.com/scrapfly/scrapfly-cli/blob/main/agent-onboarding/SKILL.md) ##### Stable JSON Envelope Every command returns the same top-level structure. Your agent never has to guess the shape, and failure modes are first-class: `success: false` + `error.code` + `error.message`. No exceptions thrown to stdout, no log lines to parse. { "success": true, "product": "scrape", "data": "..." } --- INSTALLATION## Install in Seconds One curl command - downloads a pre-compiled binary for your OS and architecture. $ `curl -fsSL https://scrapfly.io/scrapfly-cli/install | sh` Installs to `/usr/local/bin/scrapfly` by default. Works on macOS (Apple Silicon + amd64), Linux (amd64 + arm64), and Windows (amd64). After installation, authenticate once: $ `scrapfly auth login` --- QUICK START## Three Commands to Get Started From install to results in under a minute. quick start \# 1. Scrape a page and get clean markdown scrapfly scrape https://web-scraping.dev/products --format markdown { "success": true, "product": "scrape", "data": "# Products ..." } \# 2. Extract structured data with a prompt scrapfly extract --url https://web-scraping.dev/product/1 \ --prompt "product name, price, whether in stock" { "success": true, "product": "extract", "data": { "name": "Widget Pro", "price": 49.99, "in_stock": true } } \# 3. Let the agent handle a multi-step task scrapfly agent "collect prices from the first page of web-scraping.dev/products" Planning ... fetching listing ... extracting 24 products ... Done. Results: [ { "name": "...", "price": ... }, ... ] [Developer Docs](https://scrapfly.io/docs/cli) [Get Free API Key](https://scrapfly.io/register) --- FAQ## Frequently Asked Questions ### What is Scrapfly CLI? Scrapfly CLI is the official command-line interface for the Scrapfly platform. It exposes every Scrapfly product - scrape, screenshot, extract, crawl, classify, batch, and browser - as a single binary that outputs stable JSON. You can use it interactively, pipe it in shell scripts, or wire it as a tool call for any LLM agent framework. ### Do I need a Scrapfly account? Yes. The CLI calls the Scrapfly API. You need an API key, which you can get for free at scrapfly.io - 1,000 credits, no credit card required. Authenticate once with `scrapfly auth login` or set the `SCRAPFLY_API_KEY` environment variable. ### How does agent mode work? `scrapfly agent ""` sends your task description to a configured LLM provider (Anthropic, OpenAI, Gemini, or Ollama). The LLM plans the steps and calls Scrapfly tools - scrape, extract, screenshot, etc. - until the task is complete. Results are returned as JSON. You control which provider to use via `--provider` or a config file. ### Is the CLI open source? Yes. The CLI is open source under the MIT license. Source code and pre-compiled releases for Linux, macOS, and Windows are available on GitHub at [github.com/scrapfly/scrapfly-cli](https://github.com/scrapfly/scrapfly-cli). ### How do I update the CLI? Run `scrapfly update`. The CLI checks for a newer release on GitHub and replaces itself in place. No re-running the install script needed. --- // GET STARTED### Start scraping from the terminal in under a minute. Free account, 1,000 credits, no credit card. Every Scrapfly product available from a single binary. $ `curl -fsSL https://scrapfly.io/scrapfly-cli/install | sh` [ Get Free API Key ](https://scrapfly.io/register) [Read documentation](https://scrapfly.io/docs/cli)