agent-systems — MCP server from the wire protocol up

An MCP server implemented by hand over raw JSON-RPC 2.0 stdio — the initialize handshake, tools/list, tools/call, error codes — with message-level conformance tests asserting on actual wire bytes.

It exposes a real system: the hybrid retrieval service and evaluation harness from retrieval-lab. An MCP server is only interesting if the thing on the other side of the protocol is worth reaching.

Why it matters: MCP is the de-facto standard for connecting tools to models, and "can you write an MCP server" is now a real interview question. Writing it from the spec rather than the SDK is what makes the follow-up — "walk me through the handshake" — answerable.

---

Results

Check	Result
Conformance tests (wire-level)	35 passed
End-to-end stdio subprocess tests	5 passed
Total	40 passed in 0.83s
Protocol versions supported	2025-06-18, 2025-03-26, 2024-11-05
Tools exposed	4

A live session, driven by piping JSON into the process:

initialize -> {'name': 'retrieval-lab-mcp', 'version': '0.1.0'} 2025-06-18
tools/list -> ['search_documents', 'corpus_info', 'run_evaluation', 'search_my_repos']

The three spec rules most implementations get wrong

Each has a test that fails if the rule is broken:

Rule	Why it bites	Test
Never reply to a notification	notifications/initialized has no id. Replying makes a strict client error on an unsolicited response — and it surfaces one message later, so it looks like a bug in whatever came next	test_notifications_are_never_answered
The field is inputSchema, not input_schema	Snake_case makes every client see a tool with no parameters. Fails silently	test_tools_list_uses_camelCase_inputSchema
A failing tool is a successful RPC	isError: true inside the result means the tool ran and reported a problem the model can react to. A JSON-RPC error means the protocol failed. Conflating them denies the model its self-correction	test_a_failing_tool_is_a_successful_rpc_with_isError

The stdio trap

stdout is the transport. A stray print() — yours, or a library's — lands mid-stream and corrupts the session with a parse error pointing at the protocol rather than at the print.

Two defences: every diagnostic goes to stderr, and sys.stdout is redirected while tool handlers run so a third-party library that prints cannot break the channel. test_a_tool_printing_to_stdout_does_not_corrupt_the_stream registers a tool that deliberately prints and asserts the wire bytes stay clean.

---

Quick start

git clone https://github.com/harsha135135/retrieval-lab.git
git clone https://github.com/harsha135135/agent-systems.git
cd agent-systems
uv sync --extra dev

uv run pytest -q                 # 40 passed
uv run python -m agent_systems --list-tools

Drive a session by hand — no client needed:

printf '%s\n%s\n%s\n' \
 '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","clientInfo":{"name":"demo"}}}' \
 '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
 '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
 | uv run python -m agent_systems

Register it in Claude Code

.mcp.json ships in the repo:

claude mcp list        # retrieval-lab shows as connected

Then ask Claude Code to "search the arxiv corpus for papers on distributed training" or "run the retrieval evaluation with filters off".

---

The tools

Tool	What it does
search_documents	Hybrid retrieval — hard metadata filters plus semantic vectors — over 340 real arXiv papers or a synthetic listings corpus
corpus_info	Size, filterable fields, and whether the data is real or synthetic (surfaced deliberately — a model reasoning over it should know)
run_evaluation	Scores the retriever live: recall@k with its ceiling, precision, MRR, latency. Toggle apply_filters to run the ablation
search_my_repos	Read-only grep across the local repos

run_evaluation is the one worth pointing at: the model can measure the retrieval system it is querying, in the same session.

Every file, and why it exists

Path	Responsibility
protocol.py	Framing, JSON-RPC 2.0 envelope, error codes, version negotiation
server.py	Method dispatch, lifecycle rules, the stdio loop, stdout guarding
tools.py	The four tools; imports retrieval-lab lazily so its absence is a readable tool error, not a startup crash
docs/handshake.md	The handshake, message by message, with the traps
tests/unit/test_conformance.py	35 wire-level tests — assert on bytes, not internal state
tests/integration/test_stdio_session.py	5 tests against a real subprocess: pipes, framing, flushing, recovery

---

What this does not do

• No resources or prompts. Only tools is implemented, and capabilities advertise only that — declaring more would make clients call methods that 404.
• No SDK rebuild yet. The spec calls for rebuilding on the official SDK and diffing the two; the raw implementation and its conformance suite came first because that is where the understanding is.
• stdio transport only. No HTTP/SSE.
• No auth. Local stdio server; the trust boundary is the process.

---

Troubleshooting

Problem	Fix
retrieval-lab is not importable	Clone it beside this repo, then uv sync. The protocol layer works without it
Client hangs after connecting	Usually a reply sent to a notification — check id handling
Tools show with no parameters	input_schema instead of inputSchema
Garbled JSON / parse errors	Something printed to stdout. Diagnostics belong on stderr
claude mcp list shows disconnected	Run uv run python -m agent_systems directly; startup errors go to stderr