Prerequisites

Before getting started, make sure you have the following:

• Visual Studio Code installed
• Node.js 18+ (for npx command)
• Anthropic API key for Cline (separate from Scrapfly)
• Your Scrapfly API key (only if not using OAuth2)

Setup Instructions

Connect Scrapfly to Cline to enable autonomous web scraping during AI-assisted coding. Setup takes less than 5 minutes.

1. Install Cline Extension

   Install the Cline extension from the VS Code marketplace:

   From VS Code:

   • Press Ctrl+Shift+X (Windows/Linux) or ⌘+Shift+X (macOS)
   • Search for "Cline"
   • Click "Install" on the extension by saoudrizwan

   Command Line:

   code --install-extension saoudrizwan.claude-dev
   Tip: Configure Anthropic API Key

   Cline requires an Anthropic API key to function. After installing, click the Cline icon in the sidebar and follow the setup wizard to add your Anthropic API key.

2. Add Scrapfly MCP Server

   Cline uses VS Code's MCP settings. Open your VS Code settings:

   • Press Ctrl+, (Windows/Linux) or ⌘+, (macOS)
   • Search for "MCP Servers"
   • Click "Edit in settings.json"

   Select your preferred authentication method:

   {
     "mcpServers": {
       "scrapfly": {
         "command": "npx",
         "args": [
           "mcp-remote",
           "https://mcp.scrapfly.io/mcp"
         ]
       }
     }
   }

   How it works: Cline displays an OAuth2 URL in VS Code. Click it to authorize in your browser - happens once, then managed automatically. See project-scoped setup for team collaboration.

   {
     "mcpServers": {
       "scrapfly": {
         "command": "npx",
         "args": [
           "mcp-remote",
           "https://mcp.scrapfly.io/mcp?key="
         ]
       }
     }
   }

   Sign up for free to get your API key.
3. Reload VS Code Window

   Apply the MCP configuration by reloading:

   • Press Ctrl+Shift+P (Windows/Linux) or ⌘+Shift+P (macOS)
   • Type "Reload Window" and press Enter
   Tip: Check MCP Connection

   After reloading, Cline will show available MCP tools in the chat interface. Look for Scrapfly tools in the list.

4. Verify the Integration

   Confirm Scrapfly MCP is connected to Cline:

   1. Click the Cline icon in the VS Code sidebar
   2. Start a new task or chat
   3. Try a prompt: "Use Scrapfly to scrape https://news.ycombinator.com and save the top 5 posts to a file"
   4. Cline should autonomously call Scrapfly MCP tools to complete the task
   Pro Tip: Cline can autonomously chain multiple operations. For example, it can scrape data, parse it, and write code to process it - all in one request!

Example Prompts

Troubleshooting

Next Steps

• Explore available MCP tools and their capabilities
• See real-world examples of what you can build
• Learn about authentication methods in detail
• Read the FAQ for common questions