screaming-frog-mcp

MCP server to analyze Screaming Frog SEO Spider crawl files directly from an AI agent. No credentials — it only needs the path to a saved crawl file.

Purpose

Reads Screaming Frog crawl exports (.dbseospider, .seospider, .duckdb, or a folder of exported CSVs) without manually exporting anything. Built on top of Screaming Frog's ~628 pre-computed report tabs, so counts are instant even on 100K+ URL crawls — nothing gets loaded fully into memory unless you explicitly sample or export it.

Requirements

• Screaming Frog SEO Spider installed
• Java 8+ available (java -version)
• Python package screamingfrog (pip install screamingfrog)

Tools (11)

Tool	What it does
sf_full_audit	Full SEO audit in one call: HTTP errors, redirects, titles, meta descriptions, H1s, indexability, images, thin content — counts + sample URLs + a health score
sf_issues_summary	Instant counts of every issue type (no memory load, works on huge crawls)
sf_get_issue	Affected URLs for one specific issue, with pagination (limit/offset)
sf_export	Export all URLs for an issue to CSV (no row limit)
sf_list_tabs	List all available report tabs in the crawl, optionally filtered
sf_crawl_summary	General crawl summary: total URLs, status code distribution
sf_sql	Run a SQL query directly against the crawl (DuckDB, table APP.URLS)
sf_search_pages	Search a term across URL, title and meta description
sf_inlinks	All internal links pointing to a specific URL
sf_redirect_chains	Redirect chains with N or more hops
sf_compare_crawls	Diff two crawls: status, title and redirect changes

sf_get_issue/sf_export accept either a friendly alias (sin_titulo, errores_4xx, h1_missing...) or the exact Screaming Frog tab name — use sf_list_tabs to discover what's available in your crawl.

Workflow

1. Open Screaming Frog and crawl the site.
2. File → Save Crawl → save as .seospider anywhere on disk.
3. Point the agent at it:

   "Run a full audit on /path/to/crawl-2026-08.seospider"

Supported file formats

Extension	Notes
.seospider	Native Screaming Frog format — needs internal conversion (slower on first load)
.dbseospider	Same, with a DuckDB cache included — instant on second load
.duckdb	Exported DuckDB database
Folder of CSVs	Manual Screaming Frog exports

Setup

No credentials needed.

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

MCP client configuration

{
  "mcpServers": {
    "screaming-frog": {
      "command": "/path/to/.venv/bin/python3",
      "args": ["/path/to/screaming-frog-mcp/server.py"]
    }
  }
}

Notes

• First load of a .seospider file can take 30–60s (converts Derby → DuckDB internally). Subsequent loads of the same file are fast thanks to the on-disk .duckdb cache.
• Results are capped (100–500 rows depending on the tool) to avoid flooding the agent's context — use sf_export when you need the full list.
• If you have a JDK 21 installed at ~/.local/jdk21, it's picked up automatically via JAVA_HOME.

License

MIT — see LICENSE.