provenance-mcp

Answers "why does this code exist" for a file:line or a natural-language question in colinhacks/zod, by tracing code → git blame → commit → PR → linked issue. Every answer is either quoted from real history with a citation, or explicitly marked NOT RECOVERABLE -- never a plausible-sounding guess.

v1 is hardcoded to a single demo repo (colinhacks/zod). No multi-repo support, no config system.

Architecture

• packages/core -- shared TypeScript retrieval logic: local git blame/history (including pickaxe search and full non-line-scoped file history), GitHub REST enrichment (PR/issue linkage, review comments, discussion threads, repo-scoped issue/PR search), reading current file content (README/docs/changelogs), hunk-based diff chunking, commit-importance scoring, Upstash Vector indexing/search, and Report/citation assembly. All tracing logic lives here exactly once.
• packages/mcp-server -- a thin MCP server (@modelcontextprotocol/sdk) exposing four tools: trace_line, ask_why, search_history, investigate. Calls into core only.
• packages/demo -- a small live web app comparing provenance-mcp's answers against raw git log --grep, a web-search-equipped Claude call, and provenance-mcp's own multi-hop mode, side by side. See Demo below.
• packages/skill -- SKILL.md, a Claude Code Skill that drives the MCP tools and renders the fixed WHY / EVIDENCE / STILL LIVE? / VERDICT / CONFIDENCE verdict block.
• scripts/index-repo.ts -- one-off script that clones zod locally, walks its full commit history, and upserts commit documents into Upstash Vector.

trace_line/ask_why/search_history never call an LLM -- core returns structured, cited JSON, and the calling agent (Claude, via the Skill's procedure) is the one that renders prose. investigate is the one exception: it runs a bounded, LLM-orchestrated multi-hop loop (see below), but the model only ever chooses where to look next -- the actual cited text is still deterministically re-extracted from real fetched sources by the same evidence.ts logic the other tools use. There's no code path where model-authored prose becomes a citation.

The investigate tool (multi-hop mode)

trace_line/ask_why take a single deterministic pass: blame or vector-search to one anchor commit, follow its linked PR/issue, done. That's fast and cheap, but it can miss things a single pass can't reach -- a rationale stated in a review comment rather than the PR body, a fix that requires retrying the search with different words, or a design decision documented in current prose (a changelog, a migration guide) rather than any single commit.

investigate runs an autonomous loop (default: up to 40 turns / 60 tool calls / 4 minutes, Opus-tier model) with 11 read-only navigation tools -- blame_line, get_commit, get_commit_diff, pickaxe_search, file_history, vector_search, get_issue_or_pr, get_pr_review_comments, get_discussion_comments, search_issues, read_doc -- plus two terminal tools, submit_finding and give_up. Every navigation tool result carries a harness-minted sourceId tied to real fetched text; the model can only ever point at IDs it has actually seen. When it calls submit_finding, that's a pointer, not an answer -- the harness re-reads that exact source's real text through the same findRationale/buildReason logic the deterministic tools use, and if the model's chosen source doesn't actually state a reason, the report honestly comes back partial/not_recoverable regardless of what the model claimed.

Requires ANTHROPIC_API_KEY. Prefer trace_line/ask_why first; only reach for investigate when they come back partial or not_recoverable.

Setup

npm install
cp .env.example .env
# fill in GITHUB_TOKEN, UPSTASH_VECTOR_REST_URL, UPSTASH_VECTOR_REST_TOKEN
# ANTHROPIC_API_KEY is optional -- only needed for `investigate` and the demo's
# Exhibits B/D. trace_line/ask_why/search_history work without it.

Create the Upstash Vector index with a built-in embedding model (e.g. mxbai-embed-large-v1) so text can be upserted/queried directly without a separate embeddings step.

Build all workspaces:

npm run build

Index the demo repo (one-off; clones colinhacks/zod into ./.cache/zod, ~2,900 commits):

npm run index-repo

Run the MCP server (stdio transport):

npm run mcp-server

Point an MCP client (Claude Code, the MCP inspector, etc.) at packages/mcp-server/dist/index.js, and install packages/skill/SKILL.md as a Claude Code Skill to get the cited verdict format end to end.

Tools

Tool	Input	Output
trace_line	{ file, line, ref? }	Report -- precise file:line trace
ask_why	{ question }	Report, with candidates[] if the match is ambiguous
search_history	{ query, path?, topK?, context? }	ranked Citation[] from the vector index
investigate	{ file?, line?, question?, ref?, maxTurns? }	Report, with investigationTrail[] showing the search path. Requires ANTHROPIC_API_KEY; provide either file+line or question

See packages/core/src/types.ts for the full Report/Citation schema.

Demo

npm run demo

Serves a live page at http://localhost:5173 comparing four ways of answering the same question about colinhacks/zod:

• Exhibit A -- raw git log --grep, no synthesis: what a developer gets from blind keyword search.
• Exhibit B -- a live Claude call with the real web_search tool, but no access to this repo's own git history / provenance-mcp index. The fair steelman: general web search vs. specialized primary-source tracing.
• Exhibit C -- provenance-mcp's ask_why (the single-shot deterministic tool).
• Exhibit D -- provenance-mcp's investigate (multi-hop), auto-escalated to whenever C doesn't come back fully recovered.

The pre-loaded example questions are verified, not cherry-picked: one case where the fast tier alone beats web search outright, one where web search states something false with full confidence and only multi-hop corrects it, and one pulled from a real (at the time, unanswered) open GitHub issue. See the comments in packages/demo/src/examples.ts for exactly what was verified about each.

Testing

npm test

Runs Node's built-in test runner (node:test, no extra dependency) over packages/core's pure logic -- rationale extraction, confidence scoring, citation builders, commit-importance heuristics, diff chunking, and investigate's caching/ordering helpers. Two of the test cases are direct regression tests for real bugs found and fixed during development (a keyword-relevance threshold bug in findRationale, and a cache-upgrade bug in investigate's commit caching) -- see the comments in packages/core/src/investigate.test.ts for what they'd have caught.