Cursor

Cursor logo

AI-first code editor built on VS Code with native MCP support. Perfect for developers who want seamless web scraping capabilities integrated into their coding workflow.

AI Code Editor Windows macOS Linux Official Website

Prerequisites

Before getting started, make sure you have the following:

Setup Instructions

Cursor has native MCP support built-in. Follow these steps to connect Scrapfly in under 5 minutes.

  1. Open Cursor Settings

    Access the MCP configuration settings in Cursor:

    • Press Ctrl+, (Windows/Linux) or ⌘+, (macOS) to open Settings
    • Search for "MCP" in the settings search bar
    • Click "Edit in settings.json" under "MCP: Servers"
    Tip: Quick Access to settings.json

    You can also press Ctrl+Shift+P (Windows/Linux) or ⌘+Shift+P (macOS), then type "Preferences: Open User Settings (JSON)" and press Enter.

  2. Add Scrapfly MCP Configuration

    Cursor supports MCP servers in the same way as VS Code. Select your authentication method:

    How it works: On first connection, Cursor displays an OAuth2 URL in chat. Click it to authorize in your browser - happens once, then managed automatically. See project-scoped setup for team collaboration.
    Sign up for free to get your API key.
  3. Reload Cursor

    Reload the window to apply the MCP configuration:

    • Press Ctrl+Shift+P (Windows/Linux) or ⌘+Shift+P (macOS)
    • Type "Developer: Reload Window" and press Enter
    Alternative: Restart Cursor

    You can also completely close Cursor and reopen it to apply the changes.

  4. Verify the Integration

    After reloading, verify that Scrapfly MCP tools are available:

    1. Open Cursor AI chat (Ctrl+K or ⌘+K)
    2. Look for the hammer icon (🔨) indicating MCP tools are loaded
    3. Try a test prompt: "Use Scrapfly to scrape https://news.ycombinator.com and show the top posts"
    Pro Tip: Cursor's AI will automatically call the scraping_instruction_enhanced tool to get the required "pow" parameter before making scraping requests. Just describe what you want to scrape!

Example Prompts

Code Generation
Scrape HN and generate TypeScript interfaces
Data Extraction
Scrape https://web-scraping.dev/products and create comparison table
AI-Assisted Debugging
Scrape API status and help debug response format
Multi-Step Workflow
Scrape HN homepage, find top post, then scrape its comments

Troubleshooting

  • Verify npx is available: run npx --version in Cursor's integrated terminal
  • Check settings.json for valid JSON syntax (Cursor will highlight errors)
  • Reload the window: Ctrl+Shift+P → "Developer: Reload Window"
  • Check Cursor logs: Help → Toggle Developer Tools → Console tab

Access Developer Tools:

  • Press Ctrl+Shift+I (Windows/Linux) or ⌘+Option+I (macOS)
  • Go to the Console tab
  • Look for errors containing "MCP" or "scrapfly"
  • Manually copy the OAuth2 URL from Cursor chat and paste into your browser
  • Ensure you're logged into Scrapfly in your default browser
  • Try using API key authentication as a fallback
  • Check that your default browser is set correctly in system settings

If you need to manually edit settings:

Windows: %APPDATA%\Cursor\User\settings.json

macOS: ~/Library/Application Support/Cursor/User/settings.json

Linux: ~/.config/Cursor/User/settings.json

Next Steps

Summary