Agent Builders

View as markdown

Copy for LLM

Copy for LLM Open in ChatGPT Open in Claude Open in Perplexity

No-code platforms for building AI agents. Integrate Scrapfly web scraping into your visual AI workflows without writing code through API calls and custom tools.

No-Code Platforms Relevance AI Flowise LangFlow Dify Official Website

Prerequisites

Before getting started, make sure you have the following:

• Account on a no-code AI platform (Relevance AI, Flowise, LangFlow, or Dify)
• Your Scrapfly API key

Setup Instructions

Follow these platform-specific guides to integrate Scrapfly with popular no-code AI agent builders.

1. Relevance AI Integration

   Create a custom tool in Relevance AI that calls Scrapfly's API:

   1. Log in to Relevance AI
   2. Navigate to "Tools" → "Custom Tools"
   3. Click "Create New Tool"
   4. Configure the tool with these settings:

   Tool Configuration:

   {
     "name": "Scrape Webpage",
     "description": "Scrape content from a webpage using Scrapfly",
     "input_schema": {
       "url": {
         "type": "string",
         "description": "The URL to scrape",
         "required": true
       },
       "format": {
         "type": "select",
         "options": ["markdown", "text", "clean_html"],
         "default": "markdown"
       }
     },
     "api_endpoint": "{{ public_api_endpoint }}/scrape",
     "method": "GET",
     "params": {
       "key": "",
       "url": "{{input.url}}",
       "format": "{{input.format}}"
     }
   }
   Sign up for free to get your API key.
   Tip: Response Mapping

   Configure the response mapping to extract content:

   • Output Field: content
   • JSON Path: result.content
2. Flowise Integration

   Add Scrapfly as a custom tool in your Flowise workflow:

   1. Open your Flowise project
   2. Drag a "Custom Tool" node onto the canvas
   3. Connect it to your LLM node
   4. Configure the custom tool:

   Tool Code (JavaScript):

   const axios = require("axios");
   
   class ScrapflyTool {
     name = "scrape_webpage";
     description = "Scrape content from a webpage using Scrapfly";
   
     async _call(url) {
       const response = await axios.get("{{ public_api_endpoint }}/scrape", {
         params: {
           key: "",
           url: url,
           format: "markdown"
         }
       });
   
       return response.data.result?.content || "Failed to scrape";
     }
   }
   
   module.exports = { nodeClass: ScrapflyTool };
   Tip: Error Handling

   Add try-catch blocks to handle API errors gracefully:

   try {
     const response = await axios.get(...);
     return response.data.result?.content;
   } catch (error) {
     return `Error: ${error.message}`;
   }
3. LangFlow Integration

   Use the HTTP Request component to call Scrapfly:

   1. Open your LangFlow workflow
   2. Add an "HTTP Request" component
   3. Configure the component:

   HTTP Request Configuration:

   • Method: GET
   • URL: https://api.scrapfly.io/scrape
   • Parameters:
     • key: __API_KEY__
     • url: {input_url}
     • format: markdown
   • Output Path: result.content

   Connect the HTTP Request output to your LLM chain for processing.

   Tip: Dynamic URLs

   Use LangFlow variables to pass URLs dynamically from user input or previous nodes.

4. Dify Integration

   Add Scrapfly as an API tool in Dify:

   1. Go to Dify workspace
   2. Navigate to "Tools" → "API Tools"
   3. Click "Create API Tool"
   4. Fill in the tool details:

   API Tool Configuration:

   {
     "name": "Scrapfly Web Scraper",
     "description": "Scrape web content",
     "schema": {
       "url": {
         "type": "string",
         "description": "URL to scrape"
       }
     },
     "api": {
       "method": "GET",
       "url": "{{ public_api_endpoint }}/scrape",
       "params": {
         "key": "",
         "url": "{url}",
         "format": "markdown"
       }
     },
     "response_mapping": "result.content"
   }

   Add the tool to your agent or workflow, and it will be available for the LLM to call.

Example Workflows

Troubleshooting

Next Steps

• Explore available MCP tools and their capabilities
• See real-world examples of what you can build
• Learn about authentication methods in detail
• Read the FAQ for common questions