Context Catalog MCP

A miniature version of an enterprise data catalog — schema, lineage, and data-quality "trust signals" — exposed as an MCP (Model Context Protocol) server, so an AI agent can answer questions about a dataset grounded in real metadata instead of guessing.

Why this project exists

Most AI-on-data demos stop at "ask a question, get a SQL answer." That's not where the hard problem is. The hard problem is: does the agent actually know what the data means, where a number came from, and whether it can be trusted — or is it confidently making it up?

This project builds the layer that answers that:

• Schema + semantics — every table/column has a human-authored description, owner, and tags, separate from the raw data.
• Lineage — derived metrics (like customer_ltv) expose their exact source columns and transformation logic. Raw columns are explicitly labeled as raw, not silently assumed.
• Trust signals — every table gets a computed null rate, duplicate rate, freshness, and a High/Medium/Low trust label, from real checks against the data, not hardcoded values.
• Refuses to hallucinate — every tool returns found: False with a clear message when something doesn't exist, instead of inventing a plausible-looking answer. This is tested explicitly in the eval harness.

Architecture

data/seed_data.py      → builds a small SQLite DB (customers, products,
                          orders, order_items) + a catalog_metadata table
                          (the semantic layer: descriptions, owners, lineage)

catalog_core.py         → the actual logic: schema lookup, lineage lookup,
                          data-quality scoring, catalog search. Plain
                          Python, no MCP dependency — single source of
                          truth used by everything else below.

catalog_server.py       → wraps catalog_core as 4 MCP tools, so any
                          MCP-compatible client (Claude Desktop, a custom
                          agent, etc.) can call them.

eval_harness.py         → 10 deterministic test cases against catalog_core
                          directly (no LLM, no API key needed) — checks
                          correctness AND checks that unknown
                          tables/columns are correctly refused rather
                          than hallucinated.

demo_client.py           → shows what an agent would see when calling the
                          tools for 5 realistic questions. Zero setup.

agent_demo.py            → optional: a real Claude-powered agent that takes
                          a plain English question, decides which tool(s)
                          to call, and answers grounded in the result.
                          Needs ANTHROPIC_API_KEY; everything else above
                          works without one.

How to run it

Requires Python 3.10+.

pip install -r requirements.txt
./run_demo.sh

That single command will:

1. Seed the SQLite database with sample data (including intentional data quality issues, so the trust scoring has something real to catch).
2. Run the eval harness and print a pass/fail report with accuracy %.
3. Run the demo client, showing 5 realistic agent questions answered from the catalog.

Run the real MCP server

python3 catalog_server.py

To connect it to Claude Desktop, add to your MCP config:

{
  "mcpServers": {
    "context-catalog": {
      "command": "python3",
      "args": ["/absolute/path/to/catalog_server.py"]
    }
  }
}

Try the live LLM agent (optional)

export ANTHROPIC_API_KEY=sk-ant-...
python3 agent_demo.py "What does customer_ltv mean and can I trust the orders table?"

Eval results

10/10 passed  (100.0% accuracy)

Covers: correct schema/lineage/quality lookups for known entities, AND correct refusal (found: False) for unknown tables, unknown columns, and nonsense search queries — the hallucination-avoidance behavior is tested just as rigorously as the happy path.

Tech stack

Python, SQLite, the official mcp SDK, Anthropic API (optional, for the live agent demo). No external services, no cloud dependency — runs fully offline except for the optional agent demo.