urlscan-mcp

urlscan.io as MCP tools, sized so an investigation can actually run inside a context window.

The problem

One urlscan result document is 5.2 MB of request timings, response headers and cookie values. Four MCP servers for urlscan already exist. Each wraps the API endpoint for endpoint and hands that document to the model whole, which spends most of a context window to answer "is this phishing".

Measured end to end on a real scan, 525 requests across 102 domains:

	bytes
what urlscan sends	5,215,387
the same document as compact JSON	3,426,957
what one get_scan_result call puts in the agent's context	13,092

398× smaller.

13,092 is the honest figure and it is larger than the object itself. The shaped model is 5,528 bytes. An MCP result carries structured content and a text rendering of the same object. So the agent receives roughly twice the model size, and quoting 5,528 would be measuring the wrong thing.

The ceiling is enforced, not hoped for. shape_result serialises its output and halves the widest indicator list until it fits. Before that, a one-million-character page title produced a 1.5 MB "summary". test_shaped_result_fits_a_context_window fails the build against the same 5.2 MB document, stored in tests/fixtures/result_live.json.gz.

Tools

Tool	API key
pivot	no	Scans touching a domain, IP, ASN, hash or URL. Picks the field for you
search_scans	no	ElasticSearch query with cursor paging
get_screenshot	no	The screenshot, as an image block a vision model reads
get_scan_result	yes	Verdict, IP, ASN, TLS issuer, contacted infrastructure
get_dom	yes	Windows around a search string in the captured DOM
scan_url	yes	Submit a URL. Returns a UUID, not a result

A worked investigation

19 suspected phishing URLs, triaged 2026-08-14. 17 scanned; 2 rejected by urlscan with 400 DNS Error, confirmed NXDOMAIN.

Four of the 19 were *.icefactory.cl subdomains. Pivoting on the landing IP turned four URLs into a campaign:

pivot("186.64.119.45")        → 2,412 scans, AS52368 ZAM LTDA, CL
pivot("icefactory.cl")        → 2,451 scans, all hex-named subdomains
get_scan_result(uuid)         → "Adobe Acrobat - Secure PDF Document", pt-BR lure
get_dom(uuid, contains="<form")
    <form id="loginForm" method="POST"
          action="https://corpsanmarcello[.]com/report/pdf-reprt.php">
pivot("corpsanmarcello.com")  → 0 scans

The collector has zero urlscan coverage. It is never browsed, only posted to, so it appears in no verdict, no screenshot, and no blocklist derived from scan data. Reading the DOM was the only way to reach it.

Two pairs show why the verdict field is not the answer. a77e56.icefactory.cl scored 100 and 029d21.icefactory.cl scored 0, same IP, same kit, same minute. finityvoice.weebly.com ("Sign in to Xfinity.") scored 100 and ithelpdeskwebformnotice.weebly.com ("Outlook") scored 0, both posting to Weebly's own formSubmitAjax.php, so the credentials land in the attacker's Weebly inbox with no suspicious external POST for a scanner to flag.

One caution, learned the same afternoon: pivoting on 188.114.96.3 and 216.198.79.131 looked like shared infrastructure and was Cloudflare and Vercel anycast. An IP pivot on a CDN edge returns 10,000 unrelated results.

Why it is built this way

Errors are instructions, not statuses. A scan submitted ten seconds ago returns HTTP 404, and so does a UUID that never existed. The response cannot distinguish them. So the message says both, and says when to stop polling. An agent told "not found" abandons a scan twenty seconds from ready. An agent told "still running" about a typo polls forever. Each class in errors.py exists because the agent should do something different: poll, stop, wait for a quota window, fix a variable.

Shaped by default, raw on request. detail="full" attaches the whole document, and refuses above 1 MB because MCP delivers it twice. Hiding the raw document outright would make an agent conclude the field does not exist.

A submission is never re-sent. Reads retry on 429 and 5xx. The one POST does not. A timeout means the request arrived, so retrying burns a second submission quota and creates a scan the caller is never told about.

Indicators are escaped and quoted. An indicator lifted from an attacker-authored report cannot break out of its field. A reviewer brute-forced all 1,114,112 Unicode codepoints through the query builder: none produce an unescaped quote inside the wrapping quotes.

Results are labelled untrusted, in the data. Titles, brands, server banners and DOM text were authored by the scanned site. urlscan is a public corpus anyone can submit to, so an attacker can put chosen text in front of an analyst who never visited their page. This server also exposes scan_url, which fetches an arbitrary URL, so that text has an outbound channel available to it. Every result carries a content_warning field, because a caution in a tool description is read once while the data arrives on every call.

Quota is spent deliberately. The three key-gated tools fail before the network when no key is set, rather than spending a round trip to be told 403. scan_url defaults to unlisted: a public scan is visible to whoever operates the URL, which during a live incident tells an attacker they were caught.

What measurement changed

Each of these contradicts something documented, and none would have surfaced from reading.

Three endpoints need a key, not one. The published API docs describe result and dom as optional-auth. Both return 403 "You're not logged in!" without one.

403 is two different failures. urlscan also returns 403 for a query naming a field the account cannot search, with a message naming the field. Mapping every 403 to "missing API key" sends the agent to fix a key that is already correct.

And that error does not always arrive. Only unknown top-level fields 403. A typo in a dotted field, page.asnn, returns 200 with zero hits, which reads as "no scans match" rather than "you asked wrongly". Three of the seven fields this server emits are dotted, so each was verified against the live API.

The SDK moved. mcp 2.0.0 renamed FastMCP to MCPServer, moved Context, renamed the tool schema attributes to snake_case, and deprecated the logging capability ctx.info() uses. Code written from v1 memory would be wrong in four places.

What adversarial review broke

Four independent reviews ran against this: correctness, security, claim verification, and a second model. They found real defects, which is why they are listed.

API key sent to a redirect target	httpx strips Authorization across origins and keeps custom headers. Redirects are no longer followed
../../../user/quotas as a uuid	Normalised to a different authenticated endpoint with the key attached. UUIDs are validated before reaching a path
shape_result raised on 6 input shapes	In a function documented "never raises". Now total across 20 hostile documents
malicious: "false" read as True	bool("false") is True, on the field that decides the verdict
get_dom returned 6× max_chars	The separator between windows was never charged against the budget
A counted match absent from the returned text	U+0130, the Turkish dotted capital I, is the only codepoint whose .lower() changes length, so offsets from a lowercased copy addressed the wrong place. Turkish phishing is in scope
A failed POST was re-sent	Duplicate scan submission
A 1M-character title produced a 1.5 MB summary	Per-item caps multiplied above the ceiling
Obfuscated loopback accepted	http://127.0.0.1/ refused while http://2130706433/ passed
No response size limit	80 MB body accepted at 160 MB peak heap

The cause was one habit. The robustness tests were written from failure shapes that were imagined, so they agreed with the implementation. Mutation testing showed four deliberate defects surviving the original suite, including a naive bool() replacing the verdict coercion, because the string "false" was never fed to it.

The suite went from 132 tests to 278. Every test added corresponds to a finding someone reproduced.

Install

git clone https://github.com/Moussa93x/urlscan-mcp
cd urlscan-mcp
uv sync

Claude Code:

claude mcp add urlscan --env URLSCAN_API_KEY=your-key -- uv --directory /path/to/urlscan-mcp run urlscan-mcp

Claude Desktop, in claude_desktop_config.json:

{
  "mcpServers": {
    "urlscan": {
      "command": "uv",
      "args": ["--directory", "/path/to/urlscan-mcp", "run", "urlscan-mcp"],
      "env": { "URLSCAN_API_KEY": "your-key" }
    }
  }
}

Without URLSCAN_API_KEY the server starts, warns on stderr, and serves the three keyless tools. A free account issues a key at https://urlscan.io/user/signup: 5,000 public scans, 1,000 searches and 10,000 retrievals a day.

Variable	Default
URLSCAN_API_KEY	none	Required by scan_url, get_scan_result, get_dom
URLSCAN_BASE_URL	https://urlscan.io	Plain HTTP refused except on loopback
URLSCAN_TIMEOUT	30	Seconds per request
URLSCAN_MAX_SEARCH_SIZE	100	Ceiling on size, clamped rather than rejected
URLSCAN_MAX_RETRIES	3	Total attempts, so 1 disables retrying

Layout

tools/            one module per investigative step
  submit.py       scan_url
  result.py       get_scan_result
  search.py       search_scans
  pivot.py        pivot
  artifacts.py    get_dom, get_screenshot
shaping.py        raw document -> models. Pure, total, byte-bounded
models.py         Pydantic response models; field descriptions are the schema
client.py         HTTP, auth, retries, rate-limit headers, error mapping
indicators.py     indicator detection and query construction
errors.py         one class per action an agent should take
config.py         environment, validated at the boundary
server.py         MCPServer + lifespan; the only entry point

The client knows about HTTP, the tools know about MCP, neither knows the other's concerns. Adding the paid-tier endpoints is a new module in tools/ plus one line in tools/__init__.py.

Testing

uv sync --extra dev
uv run pytest                          # 278 tests, no network
uv run pytest --cov=src/urlscan_mcp    # 97%
uv run python tests/smoke_stdio.py     # real subprocess over stdio
URLSCAN_LIVE=1 uv run pytest -m live   # 4 tests against the real API

The offline suite runs the full MCP protocol in-process through Client(server) against httpx.MockTransport: no network, no subprocess, no port.

smoke_stdio.py spawns the console entry point as a subprocess and lists its tools, because the in-process tests cannot catch a broken script or a stray write to stdout, which would corrupt every message on the stdio transport.

The live suite submits a scan, polls it through the 404-while-running window, retrieves it, greps the DOM and pulls the screenshot.

CI runs 3.10 through 3.13, pins each interpreter explicitly, and asserts the running version matches the matrix entry. Without that, uv resolves any interpreter satisfying >=3.10 and four green jobs can all be the same Python.

Design notes, including what was considered and rejected: docs/design.md.

Licence

MIT.