# Scrapfly Documentation ## Table of Contents ### Dashboard - [Intro](https://scrapfly.io/docs) - [Project](https://scrapfly.io/docs/project) - [Account](https://scrapfly.io/docs/account) - [Workspace & Team](https://scrapfly.io/docs/workspace-and-team) - [Billing](https://scrapfly.io/docs/billing) ### Products #### MCP Server - [Getting Started](https://scrapfly.io/docs/mcp/getting-started) - [Tools & API Spec](https://scrapfly.io/docs/mcp/tools) - [Authentication](https://scrapfly.io/docs/mcp/authentication) - [Examples & Use Cases](https://scrapfly.io/docs/mcp/examples) - [FAQ](https://scrapfly.io/docs/mcp/faq) ##### Integrations - [Overview](https://scrapfly.io/docs/mcp/integrations) - [Claude Desktop](https://scrapfly.io/docs/mcp/integrations/claude-desktop) - [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code) - [ChatGPT](https://scrapfly.io/docs/mcp/integrations/chatgpt) - [Cursor](https://scrapfly.io/docs/mcp/integrations/cursor) - [Cline](https://scrapfly.io/docs/mcp/integrations/cline) - [Windsurf](https://scrapfly.io/docs/mcp/integrations/windsurf) - [Zed](https://scrapfly.io/docs/mcp/integrations/zed) - [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code) - [VS Code](https://scrapfly.io/docs/mcp/integrations/vscode) - [LangChain](https://scrapfly.io/docs/mcp/integrations/langchain) - [LlamaIndex](https://scrapfly.io/docs/mcp/integrations/llamaindex) - [CrewAI](https://scrapfly.io/docs/mcp/integrations/crewai) - [OpenAI](https://scrapfly.io/docs/mcp/integrations/openai) - [n8n](https://scrapfly.io/docs/mcp/integrations/n8n) - [Make](https://scrapfly.io/docs/mcp/integrations/make) - [Zapier](https://scrapfly.io/docs/mcp/integrations/zapier) - [Vapi AI](https://scrapfly.io/docs/mcp/integrations/vapi) - [Agent Builder](https://scrapfly.io/docs/mcp/integrations/agent-builder) - [Custom Client](https://scrapfly.io/docs/mcp/integrations/custom-client) #### Web Scraping API - [Getting Started](https://scrapfly.io/docs/scrape-api/getting-started) - [API Specification]() - [Monitoring](https://scrapfly.io/docs/monitoring) - [Customize Request](https://scrapfly.io/docs/scrape-api/custom) - [Debug](https://scrapfly.io/docs/scrape-api/debug) - [Anti Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - [Proxy](https://scrapfly.io/docs/scrape-api/proxy) - [Proxy Mode](https://scrapfly.io/docs/scrape-api/proxy-mode) - [Proxy Mode - Screaming Frog](https://scrapfly.io/docs/scrape-api/proxy-mode/screaming-frog) - [Proxy Mode - Apify](https://scrapfly.io/docs/scrape-api/proxy-mode/apify) - [(Auto) Data Extraction](https://scrapfly.io/docs/scrape-api/extraction) - [Javascript Rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - [Javascript Scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario) - [SSL](https://scrapfly.io/docs/scrape-api/ssl) - [DNS](https://scrapfly.io/docs/scrape-api/dns) - [Cache](https://scrapfly.io/docs/scrape-api/cache) - [Session](https://scrapfly.io/docs/scrape-api/session) - [Webhook](https://scrapfly.io/docs/scrape-api/webhook) - [Screenshot](https://scrapfly.io/docs/scrape-api/screenshot) - [Errors](https://scrapfly.io/docs/scrape-api/errors) - [Timeout](https://scrapfly.io/docs/scrape-api/understand-timeout) - [Throttling](https://scrapfly.io/docs/throttling) - [Troubleshoot](https://scrapfly.io/docs/scrape-api/troubleshoot) - [Billing](https://scrapfly.io/docs/scrape-api/billing) - [FAQ](https://scrapfly.io/docs/scrape-api/faq) #### Crawler API - [Getting Started](https://scrapfly.io/docs/crawler-api/getting-started) - [API Specification]() - [Retrieving Results](https://scrapfly.io/docs/crawler-api/results) - [WARC Format](https://scrapfly.io/docs/crawler-api/warc-format) - [Data Extraction](https://scrapfly.io/docs/crawler-api/extraction-rules) - [Webhook](https://scrapfly.io/docs/crawler-api/webhook) - [Billing](https://scrapfly.io/docs/crawler-api/billing) - [Errors](https://scrapfly.io/docs/crawler-api/errors) - [Troubleshoot](https://scrapfly.io/docs/crawler-api/troubleshoot) - [FAQ](https://scrapfly.io/docs/crawler-api/faq) #### Screenshot API - [Getting Started](https://scrapfly.io/docs/screenshot-api/getting-started) - [API Specification]() - [Accessibility Testing](https://scrapfly.io/docs/screenshot-api/accessibility) - [Webhook](https://scrapfly.io/docs/screenshot-api/webhook) - [Billing](https://scrapfly.io/docs/screenshot-api/billing) - [Errors](https://scrapfly.io/docs/screenshot-api/errors) #### Extraction API - [Getting Started](https://scrapfly.io/docs/extraction-api/getting-started) - [API Specification]() - [Rules Template](https://scrapfly.io/docs/extraction-api/rules-and-template) - [LLM Extraction](https://scrapfly.io/docs/extraction-api/llm-prompt) - [AI Auto Extraction](https://scrapfly.io/docs/extraction-api/automatic-ai) - [Webhook](https://scrapfly.io/docs/extraction-api/webhook) - [Billing](https://scrapfly.io/docs/extraction-api/billing) - [Errors](https://scrapfly.io/docs/extraction-api/errors) - [FAQ](https://scrapfly.io/docs/extraction-api/faq) #### Proxy Saver - [Getting Started](https://scrapfly.io/docs/proxy-saver/getting-started) - [Fingerprints](https://scrapfly.io/docs/proxy-saver/fingerprints) - [Optimizations](https://scrapfly.io/docs/proxy-saver/optimizations) - [SSL Certificates](https://scrapfly.io/docs/proxy-saver/certificates) - [Protocols](https://scrapfly.io/docs/proxy-saver/protocols) - [Pacfile](https://scrapfly.io/docs/proxy-saver/pacfile) - [Secure Credentials](https://scrapfly.io/docs/proxy-saver/security) - [Billing](https://scrapfly.io/docs/proxy-saver/billing) #### Cloud Browser API - [Getting Started](https://scrapfly.io/docs/cloud-browser-api/getting-started) - [Proxy & Geo-Targeting](https://scrapfly.io/docs/cloud-browser-api/proxy) - [Unblock API](https://scrapfly.io/docs/cloud-browser-api/unblock) - [File Downloads](https://scrapfly.io/docs/cloud-browser-api/file-downloads) - [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume) - [Human-in-the-Loop](https://scrapfly.io/docs/cloud-browser-api/human-in-the-loop) - [Debug Mode](https://scrapfly.io/docs/cloud-browser-api/debug-mode) - [Bring Your Own Proxy](https://scrapfly.io/docs/cloud-browser-api/bring-your-own-proxy) - [Browser Extensions](https://scrapfly.io/docs/cloud-browser-api/extensions) - [Native Browser MCP](https://scrapfly.io/docs/cloud-browser-api/mcp) - [DevTools Protocol](https://scrapfly.io/docs/cloud-browser-api/cdp-reference) ##### Integrations - [Puppeteer](https://scrapfly.io/docs/cloud-browser-api/puppeteer) - [Playwright](https://scrapfly.io/docs/cloud-browser-api/playwright) - [Selenium](https://scrapfly.io/docs/cloud-browser-api/selenium) - [Vercel Agent Browser](https://scrapfly.io/docs/cloud-browser-api/agent-browser) - [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use) - [Stagehand](https://scrapfly.io/docs/cloud-browser-api/stagehand) - [Billing](https://scrapfly.io/docs/cloud-browser-api/billing) - [Errors](https://scrapfly.io/docs/cloud-browser-api/errors) ### Tools - [Antibot Detector](https://scrapfly.io/docs/tools/antibot-detector) ### SDK - [Golang](https://scrapfly.io/docs/sdk/golang) - [Python](https://scrapfly.io/docs/sdk/python) - [Rust](https://scrapfly.io/docs/sdk/rust) - [TypeScript](https://scrapfly.io/docs/sdk/typescript) - [Scrapy](https://scrapfly.io/docs/sdk/scrapy) ### Integrations - [Getting Started](https://scrapfly.io/docs/integration/getting-started) - [LangChain](https://scrapfly.io/docs/integration/langchain) - [LlamaIndex](https://scrapfly.io/docs/integration/llamaindex) - [CrewAI](https://scrapfly.io/docs/integration/crewai) - [Zapier](https://scrapfly.io/docs/integration/zapier) - [Make](https://scrapfly.io/docs/integration/make) - [n8n](https://scrapfly.io/docs/integration/n8n) ### Academy - [Overview](https://scrapfly.io/academy) - [Web Scraping Overview](https://scrapfly.io/academy/scraping-overview) - [Tools](https://scrapfly.io/academy/tools-overview) - [Reverse Engineering](https://scrapfly.io/academy/reverse-engineering) - [Static Scraping](https://scrapfly.io/academy/static-scraping) - [HTML Parsing](https://scrapfly.io/academy/html-parsing) - [Dynamic Scraping](https://scrapfly.io/academy/dynamic-scraping) - [Hidden API Scraping](https://scrapfly.io/academy/hidden-api-scraping) - [Headless Browsers](https://scrapfly.io/academy/headless-browsers) - [Hidden Web Data](https://scrapfly.io/academy/hidden-web-data) - [JSON Parsing](https://scrapfly.io/academy/json-parsing) - [Data Processing](https://scrapfly.io/academy/data-processing) - [Scaling](https://scrapfly.io/academy/scaling) - [Walkthrough Summary](https://scrapfly.io/academy/walkthrough-summary) - [Scraper Blocking](https://scrapfly.io/academy/scraper-blocking) - [Proxies](https://scrapfly.io/academy/proxies) --- # Cline [ View as markdown ](https://scrapfly.io/?view=markdown) Copy for LLM Copy for LLM [ Open in ChatGPT ](https://chatgpt.com/?hints=search&prompt=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fmcp%2Fintegrations%2Fcline%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Claude ](https://claude.ai/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fmcp%2Fintegrations%2Fcline%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Perplexity ](https://www.perplexity.ai/search/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fmcp%2Fintegrations%2Fcline%20so%20I%20can%20ask%20questions%20about%20it.) Autonomous AI coding assistant for VS Code with MCP support. Cline can read files, make edits, and scrape web data to assist your development workflow. VS Code Extension Windows macOS Linux [ Official Website ](https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev) 1. [MCP Documentation](https://scrapfly.io/docs/mcp) 2. [Integrations](https://scrapfly.io/docs/mcp/integrations) 3. Cline ## Prerequisites Before getting started, make sure you have the following: - [Visual Studio Code](https://code.visualstudio.com/) installed - Node.js 18+ (for `npx` command) - Anthropic API key for Cline (separate from Scrapfly) - Your Scrapfly API key (only if not using OAuth2) ## Setup Instructions Connect Scrapfly to Cline to enable autonomous web scraping during AI-assisted coding. Setup takes less than 5 minutes. 1. **Install Cline Extension** Install the Cline extension from the VS Code marketplace: **From VS Code:** - Press Ctrl+Shift+X (Windows/Linux) or ⌘+Shift+X (macOS) - Search for "Cline" - Click "Install" on the extension by saoudrizwan **Command Line:** ``` code --install-extension saoudrizwan.claude-dev ``` Tip: Configure Anthropic API KeyCline requires an Anthropic API key to function. After installing, click the Cline icon in the sidebar and follow the setup wizard to add your Anthropic API key. 2. **Add Scrapfly MCP Server** Cline uses VS Code's MCP settings. Open your VS Code settings: - Press Ctrl+, (Windows/Linux) or ⌘+, (macOS) - Search for "MCP Servers" - Click "Edit in settings.json" Select your preferred authentication method: OAuth2 Recommended Secure API Key For automation ``` { "mcpServers": { "scrapfly": { "command": "npx", "args": [ "mcp-remote", "https://mcp.scrapfly.io/mcp" ] } } } ``` **How it works:** Cline displays an OAuth2 URL in VS Code. Click it to authorize in your browser - happens once, then managed automatically. [See project-scoped setup](https://scrapfly.io/docs/mcp/authentication#project-scoped) for team collaboration. ``` { "mcpServers": { "scrapfly": { "command": "npx", "args": [ "mcp-remote", "https://mcp.scrapfly.io/mcp?key=" ] } } } ``` [Sign up for free](https://scrapfly.io/register) to get your API key. 3. **Reload VS Code Window** Apply the MCP configuration by reloading: - Press Ctrl+Shift+P (Windows/Linux) or ⌘+Shift+P (macOS) - Type "Reload Window" and press Enter Tip: Check MCP ConnectionAfter reloading, Cline will show available MCP tools in the chat interface. Look for Scrapfly tools in the list. 4. **Verify the Integration** Confirm Scrapfly MCP is connected to Cline: 1. Click the Cline icon in the VS Code sidebar 2. Start a new task or chat 3. Try a prompt: *"Use Scrapfly to scrape https://news.ycombinator.com and save the top 5 posts to a file"* 4. Cline should autonomously call Scrapfly MCP tools to complete the task **Pro Tip:** Cline can autonomously chain multiple operations. For example, it can scrape data, parse it, and write code to process it - all in one request! ## Example Prompts ###### Autonomous Data Collection Scrape the top 10 posts from Hacker News and create a JSON file with their titles and URLs ###### Code Generation from Live Data Scrape https://web-scraping.dev/api/products and generate a TypeScript interface matching the data structure ###### Test Data Creation Scrape sample blog posts from https://web-scraping.dev/blog and create test fixtures for my unit tests ###### Documentation Extraction Scrape the documentation from https://web-scraping.dev/docs and create markdown files in the docs/ folder ## Troubleshooting ##### Cline does not detect Scrapfly MCP tools **Problem:** Scrapfly tools not appearing in Cline's available tools list **Solution:** - Ensure `npx` is available: run `npx --version` in VS Code terminal - Check `settings.json` is valid JSON (VS Code will show syntax errors) - Reload the window: Ctrl+Shift+P → "Reload Window" - Check VS Code Output panel (View → Output) and select "Cline" from dropdown ##### OAuth2 Authorization Issues **Problem:** OAuth2 link does not open or authorization fails **Solution:** - Manually copy the OAuth2 URL from Cline chat and paste into your browser - Ensure you're logged into Scrapfly in your browser - Check that your default browser is set correctly - Try using API key authentication as a fallback ##### Checking Cline Logs **View logs in VS Code:** - Open Output panel: View → Output (or Ctrl+Shift+U) - Select "Cline" from the dropdown in the top-right - Look for errors related to MCP server connection ##### MCP vs Anthropic API Key Confusion **Note:** Cline requires TWO separate API keys: - **Anthropic API key**: For Cline to function (configured in Cline settings) - **Scrapfly API key**: For web scraping (configured in MCP settings, or use OAuth2) Do not confuse these two! Cline will not work without the Anthropic key, and Scrapfly will not work without authentication. ## Next Steps - [Explore available MCP tools](https://scrapfly.io/docs/mcp/tools) and their capabilities - [See real-world examples](https://scrapfly.io/docs/mcp/examples) of what you can build - [Learn about authentication methods](https://scrapfly.io/docs/mcp/authentication) in detail - [Read the FAQ](https://scrapfly.io/docs/mcp/faq) for common questions [ Back to All Integrations ](https://scrapfly.io/docs/mcp/integrations)