# Scrapfly Documentation ## Table of Contents ### Dashboard - [Intro](https://scrapfly.io/docs) - [Project](https://scrapfly.io/docs/project) - [Account](https://scrapfly.io/docs/account) - [Workspace & Team](https://scrapfly.io/docs/workspace-and-team) - [Billing](https://scrapfly.io/docs/billing) ### Products #### MCP Server - [Getting Started](https://scrapfly.io/docs/mcp/getting-started) - [Tools & API Spec](https://scrapfly.io/docs/mcp/tools) - [Authentication](https://scrapfly.io/docs/mcp/authentication) - [Examples & Use Cases](https://scrapfly.io/docs/mcp/examples) - [FAQ](https://scrapfly.io/docs/mcp/faq) ##### Integrations - [Overview](https://scrapfly.io/docs/mcp/integrations) - [Claude Desktop](https://scrapfly.io/docs/mcp/integrations/claude-desktop) - [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code) - [ChatGPT](https://scrapfly.io/docs/mcp/integrations/chatgpt) - [Cursor](https://scrapfly.io/docs/mcp/integrations/cursor) - [Cline](https://scrapfly.io/docs/mcp/integrations/cline) - [Windsurf](https://scrapfly.io/docs/mcp/integrations/windsurf) - [Zed](https://scrapfly.io/docs/mcp/integrations/zed) - [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code) - [VS Code](https://scrapfly.io/docs/mcp/integrations/vscode) - [LangChain](https://scrapfly.io/docs/mcp/integrations/langchain) - [LlamaIndex](https://scrapfly.io/docs/mcp/integrations/llamaindex) - [CrewAI](https://scrapfly.io/docs/mcp/integrations/crewai) - [OpenAI](https://scrapfly.io/docs/mcp/integrations/openai) - [n8n](https://scrapfly.io/docs/mcp/integrations/n8n) - [Make](https://scrapfly.io/docs/mcp/integrations/make) - [Zapier](https://scrapfly.io/docs/mcp/integrations/zapier) - [Vapi AI](https://scrapfly.io/docs/mcp/integrations/vapi) - [Agent Builder](https://scrapfly.io/docs/mcp/integrations/agent-builder) - [Custom Client](https://scrapfly.io/docs/mcp/integrations/custom-client) #### Web Scraping API - [Getting Started](https://scrapfly.io/docs/scrape-api/getting-started) - [API Specification](https://scrapfly.io/docs/scrape-api/specification) - [Monitoring](https://scrapfly.io/docs/monitoring) - [Customize Request](https://scrapfly.io/docs/scrape-api/custom) - [Debug](https://scrapfly.io/docs/scrape-api/debug) - [Anti Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - [Proxy](https://scrapfly.io/docs/scrape-api/proxy) - [Proxy Mode](https://scrapfly.io/docs/scrape-api/proxy-mode) - [Proxy Mode - Screaming Frog](https://scrapfly.io/docs/scrape-api/proxy-mode/screaming-frog) - [Proxy Mode - Apify](https://scrapfly.io/docs/scrape-api/proxy-mode/apify) - [(Auto) Data Extraction](https://scrapfly.io/docs/scrape-api/extraction) - [Javascript Rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - [Javascript Scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario) - [SSL](https://scrapfly.io/docs/scrape-api/ssl) - [DNS](https://scrapfly.io/docs/scrape-api/dns) - [Cache](https://scrapfly.io/docs/scrape-api/cache) - [Session](https://scrapfly.io/docs/scrape-api/session) - [Webhook](https://scrapfly.io/docs/scrape-api/webhook) - [Screenshot](https://scrapfly.io/docs/scrape-api/screenshot) - [Errors](https://scrapfly.io/docs/scrape-api/errors) - [Timeout](https://scrapfly.io/docs/scrape-api/understand-timeout) - [Throttling](https://scrapfly.io/docs/throttling) - [Troubleshoot](https://scrapfly.io/docs/scrape-api/troubleshoot) - [Billing](https://scrapfly.io/docs/scrape-api/billing) - [FAQ](https://scrapfly.io/docs/scrape-api/faq) #### Crawler API - [Getting Started](https://scrapfly.io/docs/crawler-api/getting-started) - [API Specification](https://scrapfly.io/docs/crawler-api/specification) - [Retrieving Results](https://scrapfly.io/docs/crawler-api/results) - [WARC Format](https://scrapfly.io/docs/crawler-api/warc-format) - [Data Extraction](https://scrapfly.io/docs/crawler-api/extraction-rules) - [Webhook](https://scrapfly.io/docs/crawler-api/webhook) - [Billing](https://scrapfly.io/docs/crawler-api/billing) - [Errors](https://scrapfly.io/docs/crawler-api/errors) - [Troubleshoot](https://scrapfly.io/docs/crawler-api/troubleshoot) - [FAQ](https://scrapfly.io/docs/crawler-api/faq) #### Screenshot API - [Getting Started](https://scrapfly.io/docs/screenshot-api/getting-started) - [API Specification](https://scrapfly.io/docs/screenshot-api/specification) - [Accessibility Testing](https://scrapfly.io/docs/screenshot-api/accessibility) - [Webhook](https://scrapfly.io/docs/screenshot-api/webhook) - [Billing](https://scrapfly.io/docs/screenshot-api/billing) - [Errors](https://scrapfly.io/docs/screenshot-api/errors) #### Extraction API - [Getting Started](https://scrapfly.io/docs/extraction-api/getting-started) - [API Specification](https://scrapfly.io/docs/extraction-api/specification) - [Rules Template](https://scrapfly.io/docs/extraction-api/rules-and-template) - [LLM Extraction](https://scrapfly.io/docs/extraction-api/llm-prompt) - [AI Auto Extraction](https://scrapfly.io/docs/extraction-api/automatic-ai) - [Webhook](https://scrapfly.io/docs/extraction-api/webhook) - [Billing](https://scrapfly.io/docs/extraction-api/billing) - [Errors](https://scrapfly.io/docs/extraction-api/errors) - [FAQ](https://scrapfly.io/docs/extraction-api/faq) #### Proxy Saver - [Getting Started](https://scrapfly.io/docs/proxy-saver/getting-started) - [Fingerprints](https://scrapfly.io/docs/proxy-saver/fingerprints) - [Optimizations](https://scrapfly.io/docs/proxy-saver/optimizations) - [SSL Certificates](https://scrapfly.io/docs/proxy-saver/certificates) - [Protocols](https://scrapfly.io/docs/proxy-saver/protocols) - [Pacfile](https://scrapfly.io/docs/proxy-saver/pacfile) - [Secure Credentials](https://scrapfly.io/docs/proxy-saver/security) - [Billing](https://scrapfly.io/docs/proxy-saver/billing) #### Cloud Browser API - [Getting Started](https://scrapfly.io/docs/cloud-browser-api/getting-started) - [Proxy & Geo-Targeting](https://scrapfly.io/docs/cloud-browser-api/proxy) - [Unblock API](https://scrapfly.io/docs/cloud-browser-api/unblock) - [File Downloads](https://scrapfly.io/docs/cloud-browser-api/file-downloads) - [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume) - [Human-in-the-Loop](https://scrapfly.io/docs/cloud-browser-api/human-in-the-loop) - [Debug Mode](https://scrapfly.io/docs/cloud-browser-api/debug-mode) - [Bring Your Own Proxy](https://scrapfly.io/docs/cloud-browser-api/bring-your-own-proxy) - [Browser Extensions](https://scrapfly.io/docs/cloud-browser-api/extensions) ##### Integrations - [Puppeteer](https://scrapfly.io/docs/cloud-browser-api/puppeteer) - [Playwright](https://scrapfly.io/docs/cloud-browser-api/playwright) - [Selenium](https://scrapfly.io/docs/cloud-browser-api/selenium) - [Vercel Agent Browser](https://scrapfly.io/docs/cloud-browser-api/agent-browser) - [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use) - [Stagehand](https://scrapfly.io/docs/cloud-browser-api/stagehand) - [Vibium](https://scrapfly.io/docs/cloud-browser-api/vibium) - [Billing](https://scrapfly.io/docs/cloud-browser-api/billing) - [Errors](https://scrapfly.io/docs/cloud-browser-api/errors) ### Tools - [Antibot Detector](https://scrapfly.io/docs/tools/antibot-detector) ### SDK - [Golang](https://scrapfly.io/docs/sdk/golang) - [Python](https://scrapfly.io/docs/sdk/python) - [TypeScript](https://scrapfly.io/docs/sdk/typescript) - [Scrapy](https://scrapfly.io/docs/sdk/scrapy) ### Integrations - [Getting Started](https://scrapfly.io/docs/integration/getting-started) - [LangChain](https://scrapfly.io/docs/integration/langchain) - [LlamaIndex](https://scrapfly.io/docs/integration/llamaindex) - [CrewAI](https://scrapfly.io/docs/integration/crewai) - [Zapier](https://scrapfly.io/docs/integration/zapier) - [Make](https://scrapfly.io/docs/integration/make) - [n8n](https://scrapfly.io/docs/integration/n8n) ### Academy - [Overview](https://scrapfly.io/academy) - [Web Scraping Overview](https://scrapfly.io/academy/scraping-overview) - [Tools](https://scrapfly.io/academy/tools-overview) - [Reverse Engineering](https://scrapfly.io/academy/reverse-engineering) - [Static Scraping](https://scrapfly.io/academy/static-scraping) - [HTML Parsing](https://scrapfly.io/academy/html-parsing) - [Dynamic Scraping](https://scrapfly.io/academy/dynamic-scraping) - [Hidden API Scraping](https://scrapfly.io/academy/hidden-api-scraping) - [Headless Browsers](https://scrapfly.io/academy/headless-browsers) - [Hidden Web Data](https://scrapfly.io/academy/hidden-web-data) - [JSON Parsing](https://scrapfly.io/academy/json-parsing) - [Data Processing](https://scrapfly.io/academy/data-processing) - [Scaling](https://scrapfly.io/academy/scaling) - [Walkthrough Summary](https://scrapfly.io/academy/walkthrough-summary) - [Scraper Blocking](https://scrapfly.io/academy/scraper-blocking) - [Proxies](https://scrapfly.io/academy/proxies) --- # VS Code [ View as markdown ](https://scrapfly.io/?view=markdown) Copy for LLM Copy for LLM [ Open in ChatGPT ](https://chatgpt.com/?hints=search&prompt=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fmcp%2Fintegrations%2Fvscode%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Claude ](https://claude.ai/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fmcp%2Fintegrations%2Fvscode%20so%20I%20can%20ask%20questions%20about%20it.) [ Open in Perplexity ](https://www.perplexity.ai/search/new?q=Read%20from%20https%3A%2F%2Fscrapfly.io%2Fdocs%2Fmcp%2Fintegrations%2Fvscode%20so%20I%20can%20ask%20questions%20about%20it.) Setup Scrapfly MCP with any MCP-compatible VS Code extension. This guide covers the generic configuration pattern for connecting Scrapfly to VS Code-based AI assistants. Code Editor Windows macOS Linux [ Official Website ](https://code.visualstudio.com/) 1. [MCP Documentation](https://scrapfly.io/docs/mcp) 2. [Integrations](https://scrapfly.io/docs/mcp/integrations) 3. VS Code ## Prerequisites Before getting started, make sure you have the following: - [Visual Studio Code](https://code.visualstudio.com/) installed - An MCP-compatible VS Code extension (Claude Code, Cline, Roo Code, etc.) - Node.js 18+ (for `npx` command) - Your Scrapfly API key (only if not using OAuth2) ## Setup Instructions This guide covers the generic pattern for connecting Scrapfly MCP to any MCP-compatible VS Code extension. Setup takes less than 5 minutes. 1. **Install an MCP-Compatible Extension** First, install a VS Code extension that supports MCP (Model Context Protocol): **Popular MCP-compatible extensions:** - [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code) - Official Anthropic extension - [Cline](https://scrapfly.io/docs/mcp/integrations/cline) - Autonomous AI coding assistant - [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code) - AI pair programming extension - Or any other extension that supports MCP servers Tip: Check Extension DocumentationEach MCP-compatible extension may have specific setup requirements. Check the extension's documentation for any additional prerequisites. 2. **Locate VS Code Settings File**MCP servers are configured in VS Code's `settings.json` file. Choose your preferred access method: Settings UI Easiest Command Palette Quick Direct File Advanced 1. Press Ctrl+, (Windows/Linux) or ⌘+, (macOS) 2. Search for "MCP Servers" 3. Click "Edit in settings.json" 1. Press Ctrl+Shift+P (Windows/Linux) or ⌘+Shift+P (macOS) 2. Type "Preferences: Open User Settings (JSON)" 3. Press Enter **Windows:** **macOS:** **Linux:** Tip: User vs Workspace Settings Use **User Settings** to make Scrapfly available in all VS Code projects. Use **Workspace Settings** to configure Scrapfly for a specific project only. 3. **Add Scrapfly MCP Server**Select your preferred authentication method: OAuth2 Recommended Secure API Key For automation ``` { "mcpServers": { "scrapfly": { "command": "npx", "args": [ "mcp-remote", "https://mcp.scrapfly.io/mcp" ] } } } ``` **How it works:** Your MCP extension displays an OAuth2 URL. Click it to authorize in your browser - happens once, then managed automatically. [See project-scoped setup](https://scrapfly.io/docs/mcp/authentication#project-scoped) for team collaboration. ``` { "mcpServers": { "scrapfly": { "command": "npx", "args": [ "mcp-remote", "https://mcp.scrapfly.io/mcp?key=" ] } } } ``` [Sign up for free](https://scrapfly.io/register) to get your API key. ### Advanced: Multiple MCP Servers You can configure multiple MCP servers in the same settings file: ``` { "mcpServers": { "scrapfly": { "command": "npx", "args": ["mcp-remote", "https://mcp.scrapfly.io/mcp"] }, "another-mcp-server": { "command": "npx", "args": ["another-mcp-package"] } } } ``` 4. **Reload VS Code Window** Apply the MCP configuration by reloading: - Press Ctrl+Shift+P (Windows/Linux) or ⌘+Shift+P (macOS) - Type "Reload Window" and press Enter Alternative: Restart VS CodeYou can also completely close VS Code and reopen it to ensure all changes are applied. 5. **Verify the Integration** Confirm Scrapfly MCP is working with your extension: 1. Open your MCP-compatible extension's interface (sidebar, panel, or chat) 2. Look for the **hammer icon (🔨)** or MCP tools indicator 3. Try a test prompt: *"Use Scrapfly to scrape https://news.ycombinator.com and show the top 5 posts"* 4. Your extension should call Scrapfly MCP tools and return the results **Pro Tip:** Most MCP-compatible extensions will automatically call the `scraping_instruction_enhanced` tool to get required authentication parameters before making scraping requests. ## Example Prompts ###### Simple Page Scraping Scrape https://news.ycombinator.com and show the top 5 post titles ###### Structured Data Extraction Scrape https://web-scraping.dev/products and create a JSON array of product names and prices ###### Screenshot Capture Take a full-page screenshot of https://scrapfly.io ###### Multi-Step Workflow Scrape the Hacker News homepage, find the top post, then scrape that post's discussion page ## Troubleshooting ##### MCP tools not detected **Problem:** Extension does not detect Scrapfly MCP tools **Solution:** - Ensure `npx` is available: run `npx --version` in VS Code terminal - Check `settings.json` is valid JSON (VS Code will show syntax errors) - Reload the window: Ctrl+Shift+P → "Reload Window" - Check VS Code Output panel (View → Output) and select your extension from dropdown - Verify your extension supports MCP (check extension documentation) ##### Checking Extension Logs **View logs in VS Code:** - Open Output panel: Ctrl+Shift+U (Windows/Linux) or ⌘+Shift+U (macOS) - Select your extension name from the dropdown in the top-right - Look for errors related to MCP server connection or npx command execution ##### OAuth2 Authentication Fails **Problem:** Browser does not open for OAuth2 authentication **Solution:** - Manually copy the OAuth2 URL from extension interface and paste it into your browser - Ensure you're logged into Scrapfly in your browser - Check that your default browser is set correctly - Try using API key authentication as a fallback ##### Node.js or npx Not Found **Problem:** VS Code cannot find `npx` command **Solution:** - Ensure Node.js 18+ is installed: run `node --version` in terminal - Verify `npx` is in your PATH: run `npx --version` - Restart VS Code after installing Node.js - On Windows, you may need to add Node.js to PATH manually ##### Multiple MCP Servers Conflict **Problem:** Issues when multiple MCP servers are configured **Solution:** - Ensure each MCP server has a unique key in the `mcpServers` object - Check that there are no duplicate server names - Verify JSON syntax is correct (commas, brackets, quotes) - Try disabling other MCP servers temporarily to isolate the issue ## Next Steps - [Explore available MCP tools](https://scrapfly.io/docs/mcp/tools) and their capabilities - [See real-world examples](https://scrapfly.io/docs/mcp/examples) of what you can build - [Learn about authentication methods](https://scrapfly.io/docs/mcp/authentication) in detail - [Read the FAQ](https://scrapfly.io/docs/mcp/faq) for common questions [ Back to All Integrations ](https://scrapfly.io/docs/mcp/integrations)