MCP 服务器

toolgate-firecrawl-mcp

A recoverable Firecrawl MCP server that adds payment lifecycle, idempotency, and automatic refunds/fallbacks to web scraping, enabling reliable paid scraping via natural language.

README

toolgate-firecrawl-mcp

A recoverable Firecrawl MCP server powered by Toolgate.

Every paid scrape goes through a full lifecycle: balance check → execute → charge → trace. If it fails, you get a refund or fallback — never a silent loss.

What happens when you scrape

Agent calls scrape_url
  → Toolgate checks idempotency (duplicate? return cached)
  → Toolgate checks balance (enough? deduct $0.10)
  → Firecrawl scrapes the URL (JS-rendered, clean markdown)
  → On success: charge confirmed, trace saved
  → On failure: refund issued, trace saved with reason
  → On no balance: fallback (fetch+cheerio metadata) returned free

Quick Start

Install & run (demo mode — no API key needed)

npm install
npm run dev

Demo mode uses fetch+cheerio as the scraping backend. scrape_url and balance tools work out of the box. No Firecrawl credits consumed.

Claude Desktop — demo mode

{
  "mcpServers": {
    "toolgate-firecrawl": {
      "command": "node",
      "args": ["/path/to/toolgate-firecrawl-mcp/dist/server.js"],
      "env": {
        "TOOLGATE_DEMO_MODE": "true"
      }
    }
  }
}

Claude Desktop — production (real Firecrawl)

{
  "mcpServers": {
    "toolgate-firecrawl": {
      "command": "node",
      "args": ["/path/to/toolgate-firecrawl-mcp/dist/server.js"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-your-key-here"
      }
    }
  }
}

Tools

Tool	Price	Description
`scrape_url`	$0.10/call	Scrape a URL → clean markdown. Fallback: basic metadata free.
`web_search`	$0.01/result (min $0.05)	Web search via Firecrawl. Requires API key.
`extract_data`	$0.15/call	Structured JSON extraction with schema. Requires API key.
`check_balance`	free	Check your current Toolgate balance.
`add_balance`	free	Add demo balance (demo mode only).
`view_traces`	free	View recent execution traces with decisions and charge status.

Demo

npm run scenario

Example output:

{
  "passed": 6,
  "failed": 0,
  "skipped": 2,
  "scenarios": [
    { "name": "url_validation_ssrf", "passed": true },
    { "name": "payment_missing_fallback", "passed": true },
    { "name": "credit_balance", "passed": true },
    { "name": "paid_scrape", "passed": true },
    { "name": "duplicate_request", "passed": true },
    { "name": "unreachable_url", "skipped": true, "reason": "network" },
    { "name": "free_tier_not_applicable", "passed": true },
    { "name": "trace_output", "passed": true }
  ]
}

How Toolgate works here

Toolgate wraps every Firecrawl call with a payment lifecycle. Each call gets an idempotency key — retrying with the same key returns the cached result at no extra cost. If the Firecrawl call fails after payment was deducted, the charge is automatically refunded. Every decision is recorded in a queryable execution trace.

Environment Variables

Variable	Default	Description
`FIRECRAWL_API_KEY`	—	Firecrawl API key. Omit for demo mode.
`FIRECRAWL_API_URL`	`https://api.firecrawl.dev`	Firecrawl API base URL.
`TOOLGATE_PUBLISHER_KEY`	`tg_firecrawl_demo`	Toolgate publisher key.
`TOOLGATE_DEMO_MODE`	`true` if no API key	Force demo mode.
`TOOLGATE_PRELOAD_BALANCE`	`1.0` (demo only)	USD pre-loaded in demo mode.
`SCRAPE_TIMEOUT_MS`	`15000`	Timeout for scrape requests.
`MAX_RESPONSE_BYTES`	`5242880` (5MB)	Max response size.