# Firecrawl + Dify (/developer-guides/workflow-automation/dify)

<!-- agent-signals: reading_time_min: 10 · est_tokens: 4894 · updated: 2026-07-30 -->
Related: [Firecrawl + n8n](/developer-guides/workflow-automation/n8n.md), [Firecrawl + Zapier](/developer-guides/workflow-automation/zapier.md), [Firecrawl + Make](/developer-guides/workflow-automation/make.md), [Overview](/webhooks/overview.md)

<Note>
  **Official Dify Plugin:** [marketplace.dify.ai/plugins/langgenius/firecrawl](https://marketplace.dify.ai/plugins/langgenius/firecrawl)

  Official plugin by Dify team • 44,000+ installs • Chatflow & Agent apps • Free to use
</Note>

## Dify Integration Overview [#dify-integration-overview]

Dify is an open-source LLM app development platform. The official Firecrawl plugin enables web crawling and scraping directly in your AI workflows.

<CardGroup cols="2">
  <Card title="Chatflow & Workflow Apps" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M8 7L16 7&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M8 11L12 11&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 21.5V21C13 18.1716 13 16.7574 13.8787 15.8787C14.7574 15 16.1716 15 19 15H19.5M20 13.3431V10C20 6.22876 20 4.34315 18.8284 3.17157C17.6569 2 15.7712 2 12 2C8.22877 2 6.34315 2 5.17157 3.17157C4 4.34314 4 6.22876 4 10L4 14.5442C4 17.7892 4 19.4117 4.88607 20.5107C5.06508 20.7327 5.26731 20.9349 5.48933 21.1139C6.58831 22 8.21082 22 11.4558 22C12.1614 22 12.5141 22 12.8372 21.886C12.9044 21.8623 12.9702 21.835 13.0345 21.8043C13.3436 21.6564 13.593 21.407 14.0919 20.9081L18.8284 16.1716C19.4065 15.5935 19.6955 15.3045 19.8478 14.9369C20 14.5694 20 14.1606 20 13.3431Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Build visual pipelines with Firecrawl nodes for data extraction
  </Card>

  <Card title="Agent Applications" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M8 7L16 7&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M8 11L12 11&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 21.5V21C13 18.1716 13 16.7574 13.8787 15.8787C14.7574 15 16.1716 15 19 15H19.5M20 13.3431V10C20 6.22876 20 4.34315 18.8284 3.17157C17.6569 2 15.7712 2 12 2C8.22877 2 6.34315 2 5.17157 3.17157C4 4.34314 4 6.22876 4 10L4 14.5442C4 17.7892 4 19.4117 4.88607 20.5107C5.06508 20.7327 5.26731 20.9349 5.48933 21.1139C6.58831 22 8.21082 22 11.4558 22C12.1614 22 12.5141 22 12.8372 21.886C12.9044 21.8623 12.9702 21.835 13.0345 21.8043C13.3436 21.6564 13.593 21.407 14.0919 20.9081L18.8284 16.1716C19.4065 15.5935 19.6955 15.3045 19.8478 14.9369C20 14.5694 20 14.1606 20 13.3431Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Give AI agents the power to scrape live web data on demand
  </Card>
</CardGroup>

## Firecrawl Tools in Dify [#firecrawl-tools-in-dify]

<AccordionGroup>
  <Accordion title="Scrape" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M10 18C10 18 7.50004 16.1588 7.50003 15.5C7.50003 14.8412 10 13 10 13&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M14 18C14 18 16.5 16.1588 16.5 15.5C16.5 14.8412 14 13 14 13&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 2.5V3C13 5.82843 13 7.24264 13.8787 8.12132C14.7574 9 16.1716 9 19 9H19.5M20 10.6569V14C20 17.7712 20 19.6569 18.8284 20.8284C17.6569 22 15.7712 22 12 22C8.22876 22 6.34315 22 5.17157 20.8284C4 19.6569 4 17.7712 4 14V9.45584C4 6.21082 4 4.58831 4.88607 3.48933C5.06508 3.26731 5.26731 3.06508 5.48933 2.88607C6.58831 2 8.21082 2 11.4558 2C12.1614 2 12.5141 2 12.8372 2.11401C12.9044 2.13772 12.9702 2.165 13.0345 2.19575C13.3436 2.34355 13.593 2.593 14.0919 3.09188L18.8284 7.82843C19.4065 8.40649 19.6955 8.69552 19.8478 9.06306C20 9.4306 20 9.83935 20 10.6569Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Convert any URL into clean, structured data. Transform raw HTML into actionable insights.

    **Use Cases:** Extract product data, scrape article content, get structured data with JSON mode.
  </Accordion>

  <Accordion title="Crawl" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M8 7L16 7&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M8 11L12 11&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 21.5V21C13 18.1716 13 16.7574 13.8787 15.8787C14.7574 15 16.1716 15 19 15H19.5M20 13.3431V10C20 6.22876 20 4.34315 18.8284 3.17157C17.6569 2 15.7712 2 12 2C8.22877 2 6.34315 2 5.17157 3.17157C4 4.34314 4 6.22876 4 10L4 14.5442C4 17.7892 4 19.4117 4.88607 20.5107C5.06508 20.7327 5.26731 20.9349 5.48933 21.1139C6.58831 22 8.21082 22 11.4558 22C12.1614 22 12.5141 22 12.8372 21.886C12.9044 21.8623 12.9702 21.835 13.0345 21.8043C13.3436 21.6564 13.593 21.407 14.0919 20.9081L18.8284 16.1716C19.4065 15.5935 19.6955 15.3045 19.8478 14.9369C20 14.5694 20 14.1606 20 13.3431Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Perform recursive crawls of websites and subdomains to gather extensive content.

    **Use Cases:** Full site content extraction, documentation scraping, multi-page data collection.
  </Accordion>

  <Accordion title="Map" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M8 7L16 7&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M8 11L12 11&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 21.5V21C13 18.1716 13 16.7574 13.8787 15.8787C14.7574 15 16.1716 15 19 15H19.5M20 13.3431V10C20 6.22876 20 4.34315 18.8284 3.17157C17.6569 2 15.7712 2 12 2C8.22877 2 6.34315 2 5.17157 3.17157C4 4.34314 4 6.22876 4 10L4 14.5442C4 17.7892 4 19.4117 4.88607 20.5107C5.06508 20.7327 5.26731 20.9349 5.48933 21.1139C6.58831 22 8.21082 22 11.4558 22C12.1614 22 12.5141 22 12.8372 21.886C12.9044 21.8623 12.9702 21.835 13.0345 21.8043C13.3436 21.6564 13.593 21.407 14.0919 20.9081L18.8284 16.1716C19.4065 15.5935 19.6955 15.3045 19.8478 14.9369C20 14.5694 20 14.1606 20 13.3431Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Generate a complete map of all URLs present on a website.

    **Use Cases:** Site structure analysis, SEO auditing, URL discovery for batch scraping.
  </Accordion>

  <Accordion title="Crawl Job" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M11 6L21 6&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M11 12L21 12&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M11 18L21 18&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M3 7.39286C3 7.39286 4 8.04466 4.5 9C4.5 9 6 5.25 8 4&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M3 18.3929C3 18.3929 4 19.0447 4.5 20C4.5 20 6 16.25 8 15&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Retrieve scraping results based on a Job ID or cancel ongoing tasks.

    **Use Cases:** Monitor long-running crawls, manage async scraping workflows, cancel operations when needed.
  </Accordion>
</AccordionGroup>

## Getting Started [#getting-started]

<Steps>
  <Step title="Install Firecrawl Plugin">
    Access the [Dify Plugin Marketplace](https://marketplace.dify.ai/plugins/langgenius/firecrawl) and install the Firecrawl tool
  </Step>

  <Step title="Get Firecrawl API Key">
    Visit [Firecrawl API Keys](https://www.firecrawl.dev/app/api-keys) and create a new API key
  </Step>

  <Step title="Authorize in Dify">
    Navigate to **Plugins > Firecrawl > To Authorize** and input your API key
  </Step>

  <Step title="Add to Your Workflow">
    Drag Firecrawl tools into your Chatflow, Workflow, or Agent application
  </Step>

  <Step title="Configure & Test">
    Set up parameters and test your workflow
  </Step>
</Steps>

## Usage Patterns [#usage-patterns]

<Tabs>
  <Tab title="Chatflow Apps">
    **Visual Pipeline Integration**

    1. Add Firecrawl node to your pipeline
    2. Select action (Map, Crawl, Scrape)
    3. Define input variables
    4. Execute pipeline sequentially

    **Example Flow:**

    ```
    User Input → Firecrawl (Scrape) → LLM Processing → Response
    ```
  </Tab>

  <Tab title="Workflow Apps">
    **Automated Data Processing**

    Build multi-step workflows with:

    * Scheduled scraping
    * Data transformation
    * Database storage
    * Notifications

    **Example Flow:**

    ```
    Schedule Trigger → Firecrawl (Crawl) → Data Processing → Storage
    ```
  </Tab>

  <Tab title="Agent Apps">
    **AI-Powered Web Access**

    Give agents real-time web scraping capabilities:

    1. Add Firecrawl tool to Agent
    2. Agent autonomously decides when to scrape
    3. LLM analyzes extracted content
    4. Agent provides informed responses

    **Use Case:** Customer support agents that reference live documentation
  </Tab>
</Tabs>

## Common Use Cases [#common-use-cases]

<CardGroup cols="2">
  <Card title="AI Chatbot with Live Data" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M8 7L16 7&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M8 11L12 11&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 21.5V21C13 18.1716 13 16.7574 13.8787 15.8787C14.7574 15 16.1716 15 19 15H19.5M20 13.3431V10C20 6.22876 20 4.34315 18.8284 3.17157C17.6569 2 15.7712 2 12 2C8.22877 2 6.34315 2 5.17157 3.17157C4 4.34314 4 6.22876 4 10L4 14.5442C4 17.7892 4 19.4117 4.88607 20.5107C5.06508 20.7327 5.26731 20.9349 5.48933 21.1139C6.58831 22 8.21082 22 11.4558 22C12.1614 22 12.5141 22 12.8372 21.886C12.9044 21.8623 12.9702 21.835 13.0345 21.8043C13.3436 21.6564 13.593 21.407 14.0919 20.9081L18.8284 16.1716C19.4065 15.5935 19.6955 15.3045 19.8478 14.9369C20 14.5694 20 14.1606 20 13.3431Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Build RAG-powered chatbots that scrape and reference live website content
  </Card>

  <Card title="Content Analysis Agent" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M16.998 7.12652C17.3182 7.04393 17.654 7 18 7C20.2091 7 22 8.79086 22 11C22 13.2091 20.2091 15 18 15C17.6451 15 17.3009 14.9538 16.9733 14.867M16.998 7.12652C16.9993 7.08451 17 7.04233 17 7C17 4.79086 15.2091 3 13 3C11.0824 3 9.47994 4.34939 9.09041 6.15043M16.998 7.12652C16.9769 7.80763 16.7854 8.44584 16.4649 9M16.9733 14.867C16.9909 14.7472 17 14.6247 17 14.5C17 13.2905 16.1411 12.2816 15 12.05M16.9733 14.867C16.7957 16.0737 15.756 17 14.5 17H14C11.7909 17 10 18.7909 10 21M9.09041 6.15043C8.74377 6.05243 8.37801 6 8 6C5.79086 6 4 7.79086 4 10C4 10.3886 4.05542 10.7643 4.15878 11.1195M9.09041 6.15043C10.1015 6.43625 10.9498 7.10965 11.4649 8M4.15878 11.1195C2.9114 11.4832 2 12.6352 2 14C2 15.6569 3.34315 17 5 17C6.30622 17 7.41746 16.1652 7.82929 15M4.15878 11.1195C4.24921 11.4303 4.37632 11.7255 4.53513 12&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M11.8361 11.7435C11.3257 12.2353 10.453 12.3202 9.70713 11.9008C8.9612 11.4814 8.58031 10.6917 8.73535 10&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Agents that research topics by scraping and analyzing multiple sources
  </Card>

  <Card title="Competitor Monitoring" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M21.8551 15.5L18.9298 5.60666C18.6485 4.65457 17.7646 4 16.7601 4C15.475 4 14.4485 5.05662 14.502 6.32437L15 16M22 16.5C22 18.433 20.433 20 18.5 20C16.567 20 15 18.433 15 16.5C15 14.567 16.567 13 18.5 13C20.433 13 22 14.567 22 16.5Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M10 8H14M9 16H15&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M2.14494 15.5L5.07067 5.60666C5.35192 4.65457 6.23586 4 7.24034 4C8.52545 4 9.55194 5.05662 9.49844 6.32437L9.00044 16M9 16.5C9 18.433 7.433 20 5.5 20C3.567 20 2 18.433 2 16.5C2 14.567 3.567 13 5.5 13C7.433 13 9 14.567 9 16.5Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Automated workflows that track competitor websites and alert on changes
  </Card>

  <Card title="Data Enrichment Pipeline" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><ellipse cx=&#x22;12&#x22; cy=&#x22;5&#x22; rx=&#x22;8&#x22; ry=&#x22;3&#x22; stroke=&#x22;currentColor&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M7 10.842C7.60158 11.0229 8.27434 11.1718 9 11.282&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M20 12C20 13.6569 16.4183 15 12 15C7.58172 15 4 13.6569 4 12&#x22; stroke=&#x22;currentColor&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M7 17.842C7.60158 18.0229 8.27434 18.1718 9 18.282&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M20 5V19C20 20.6569 16.4183 22 12 22C7.58172 22 4 20.6569 4 19V5&#x22; stroke=&#x22;currentColor&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    Extract and enrich data from websites into structured databases
  </Card>
</CardGroup>

## Firecrawl Actions [#firecrawl-actions]

| Tool          | Description                    | Best For                |
| ------------- | ------------------------------ | ----------------------- |
| **Scrape**    | Single-page data extraction    | Quick content capture   |
| **Crawl**     | Multi-page recursive crawling  | Full site extraction    |
| **Map**       | URL discovery and site mapping | SEO analysis, URL lists |
| **Crawl Job** | Async job management           | Long-running operations |

## Best Practices [#best-practices]

<CardGroup cols="2">
  <Card title="Agent Apps" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M8 7L16 7&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M8 11L12 11&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 21.5V21C13 18.1716 13 16.7574 13.8787 15.8787C14.7574 15 16.1716 15 19 15H19.5M20 13.3431V10C20 6.22876 20 4.34315 18.8284 3.17157C17.6569 2 15.7712 2 12 2C8.22877 2 6.34315 2 5.17157 3.17157C4 4.34314 4 6.22876 4 10L4 14.5442C4 17.7892 4 19.4117 4.88607 20.5107C5.06508 20.7327 5.26731 20.9349 5.48933 21.1139C6.58831 22 8.21082 22 11.4558 22C12.1614 22 12.5141 22 12.8372 21.886C12.9044 21.8623 12.9702 21.835 13.0345 21.8043C13.3436 21.6564 13.593 21.407 14.0919 20.9081L18.8284 16.1716C19.4065 15.5935 19.6955 15.3045 19.8478 14.9369C20 14.5694 20 14.1606 20 13.3431Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    * Let agents decide when to scrape
    * Use natural language instructions
    * Enable tool calling in LLM settings
    * Monitor token usage with large scrapes
  </Card>

  <Card title="Workflow Apps" icon="<svg xmlns=&#x22;http://www.w3.org/2000/svg&#x22; viewBox=&#x22;0 0 24 24&#x22; fill=&#x22;none&#x22;><path d=&#x22;M8 7L16 7&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M8 11L12 11&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/><path d=&#x22;M13 21.5V21C13 18.1716 13 16.7574 13.8787 15.8787C14.7574 15 16.1716 15 19 15H19.5M20 13.3431V10C20 6.22876 20 4.34315 18.8284 3.17157C17.6569 2 15.7712 2 12 2C8.22877 2 6.34315 2 5.17157 3.17157C4 4.34314 4 6.22876 4 10L4 14.5442C4 17.7892 4 19.4117 4.88607 20.5107C5.06508 20.7327 5.26731 20.9349 5.48933 21.1139C6.58831 22 8.21082 22 11.4558 22C12.1614 22 12.5141 22 12.8372 21.886C12.9044 21.8623 12.9702 21.835 13.0345 21.8043C13.3436 21.6564 13.593 21.407 14.0919 20.9081L18.8284 16.1716C19.4065 15.5935 19.6955 15.3045 19.8478 14.9369C20 14.5694 20 14.1606 20 13.3431Z&#x22; stroke=&#x22;currentColor&#x22; stroke-linecap=&#x22;round&#x22; stroke-linejoin=&#x22;round&#x22; stroke-width=&#x22;1.5&#x22;/></svg>">
    * Use Map before Crawl for large sites
    * Set appropriate crawl limits
    * Add error handling nodes
    * Test with small datasets first
  </Card>
</CardGroup>

## Dify vs Other Platforms [#dify-vs-other-platforms]

| Feature         | Dify                 | Make                | Zapier              | n8n                 |
| --------------- | -------------------- | ------------------- | ------------------- | ------------------- |
| **Type**        | LLM app platform     | Workflow automation | Workflow automation | Workflow automation |
| **Best For**    | AI agents & chatbots | Visual workflows    | Quick automation    | Developer control   |
| **Pricing**     | Open-source + Cloud  | Operations-based    | Per-task            | Flat monthly        |
| **AI-Native**   | Yes                  | Partial             | Partial             | Partial             |
| **Self-Hosted** | Yes                  | No                  | No                  | Yes                 |

<Tip>
  **Pro Tip:** Dify excels at building AI-native applications where agents need dynamic web access. Perfect for chatbots, research assistants, and AI tools that need live data.
</Tip>