Django

Use Firecrawl with Django to scrape, search, and interact with web data in your Python web application.

Prerequisites

Django 4+ project
A Firecrawl API key — get one free

Install the SDK

pip install firecrawl-py

Add your API key to your Django settings or environment:

export FIRECRAWL_API_KEY=fc-YOUR-API-KEY

Create views

Add search, scrape, and interact views to your Django app. In views.py:

import json
import os
from django.http import JsonResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_POST
from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])


@csrf_exempt
@require_POST
def search_view(request):
    body = json.loads(request.body)
    results = firecrawl.search(body["query"], limit=body.get("limit", 5))
    return JsonResponse(
        [{"title": r.title, "url": r.url} for r in results.web],
        safe=False,
    )


@csrf_exempt
@require_POST
def scrape_view(request):
    body = json.loads(request.body)
    result = firecrawl.scrape(body["url"])
    return JsonResponse({
        "markdown": result.markdown,
        "metadata": result.metadata,
    })


@csrf_exempt
@require_POST
def interact_start_view(request):
    body = json.loads(request.body)
    result = firecrawl.scrape(body["url"], formats=["markdown"])
    return JsonResponse({"scrape_id": result.metadata.scrape_id})


@csrf_exempt
@require_POST
def interact_view(request):
    body = json.loads(request.body)
    response = firecrawl.interact(body["scrape_id"], prompt=body["prompt"])
    return JsonResponse({"output": response.output})


@csrf_exempt
@require_POST
def interact_stop_view(request):
    body = json.loads(request.body)
    firecrawl.stop_interaction(body["scrape_id"])
    return JsonResponse({"status": "stopped"})

Wire up URLs

In urls.py:

from django.urls import path
from . import views

urlpatterns = [
    path("api/search/", views.search_view),
    path("api/scrape/", views.scrape_view),
    path("api/interact/start/", views.interact_start_view),
    path("api/interact/", views.interact_view),
    path("api/interact/stop/", views.interact_stop_view),
]

Test it

python manage.py runserver

# Search the web
curl -X POST http://localhost:8000/api/search/ \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping", "limit": 5}'

# Scrape a page
curl -X POST http://localhost:8000/api/scrape/ \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Start an interactive session
curl -X POST http://localhost:8000/api/interact/start/ \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com"}'

Management command

Use Firecrawl in a Django management command for scripts and data pipelines. Create management/commands/scrape.py:

import os
from django.core.management.base import BaseCommand
from firecrawl import Firecrawl


class Command(BaseCommand):
    help = "Scrape a URL and print the markdown"

    def add_arguments(self, parser):
        parser.add_argument("url", type=str)

    def handle(self, *args, **options):
        firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])
        result = firecrawl.scrape(options["url"])
        self.stdout.write(result.markdown)

python manage.py scrape https://example.com

Next steps

Scrape docs

All scrape options including formats, actions, and proxies

Search docs

Search the web and get full page content

Interact docs

Click, fill forms, and extract dynamic content

Python SDK reference

Full SDK reference with crawl, map, async, and more