Sourcebook

Sourcebook is a git-backed markdown wiki that AI coding agents read and update over MCP.

I built it because project knowledge kept getting lost between agent sessions. How a service actually works, why we picked one approach over another, what our internal terms mean, what broke last month and why: that lived in design docs, incident writeups, old chat logs, and people's heads. Every session started cold and I re-explained the same things.

Sourcebook gives that knowledge one structured place. Agents search it before starting work, update pages when they learn something, and add pages when nothing covers the topic. Documents are validated against schemas, every change is a git commit with an author, and the tooling flags pages that have gone stale or started contradicting each other.

It is aimed at a small team on a long-running project that uses coding agents daily. The wiki is a curated, bounded set of pages that someone keeps current, not an index over every file you have.

Status: an experiment, published to see whether the approach is useful to anyone else. If you would run this, or something is missing that stops you, open a Discussion.

What a session looks like

You paste last week's incident postmortem into an ingestion session. Sourcebook proposes a plan: add the new failure mode to the affected service's system card, add a step to the incident-response process doc, define one new glossary term. Each change cites the postmortem. Nothing else is touched, because the prompts treat "already covered" as a correct answer. You apply it, and each change lands as a git commit attributed to you.

Three weeks later a teammate asks their agent how that service's retry logic works. The agent reads the system card, which arrives with a notice that its last_verified date has aged. The work the agent just did confirms the doc is still accurate, so it calls verify_document once and the clock resets, committed under the teammate's name. Nobody scheduled a documentation review.

What it stores

Six document types, each with required frontmatter and required sections:

Type	Holds
System card	How one system or service works: purpose, stack, patterns, gotchas
Process	A procedure someone follows, step by step
Decision record	An ADR: what was decided, what was rejected, the consequences
API reference	The parts of a library or framework API your team actually uses
Knowledge page	A synthesized answer worth keeping, usually promoted from a conversation
Source summary	What an ingested source said, and which pages it changed

Alongside those: a glossary (terminology as data rather than prose), conventions (hard rules agents must not break), and context bundles (a named set of pages an agent loads in one call, trimmed to a token budget).

How you interact with it

• Agents use the MCP server to search, read, write, verify, ingest, promote, and lint.
• Humans use the web UI to browse, edit, and review, with dashboards for staleness, lint findings, and recent activity.
• The CLI (sourcebook) handles setup, validation, lint, bundles, and running the servers.

All three call the same modules, so search ranking and bundle assembly cannot drift between them (ADR-002).

Try it locally

This repo ships with its own wiki, so you can browse a populated instance before building anything:

git clone <this-repo> sourcebook && cd sourcebook
pip install -e .
sourcebook serve         # this repo's own wiki at http://localhost:5000

Not on PyPI yet, so installing from a clone is the supported path. The sourcebook CLI then works from any directory.

To start your own, run this inside your project's git repository, so doc changes get history and attribution:

cd ~/code/your-project
sourcebook init          # scaffolds docs/ai/ with schemas, seed examples, glossary
sourcebook serve         # web UI on :5000
sourcebook serve-mcp     # MCP server on :8000 (second terminal)

Then point an agent at it:

claude mcp add --scope user sourcebook --transport http http://localhost:8000/mcp

Docker runs both servers under one supervisor:

docker compose up --build -d

The seed content demonstrates each document type using a small fictional product. Replace it with your own. CLAUDE.md in this repo shows the kind of agent instructions that get the wiki consulted and maintained as part of normal work.

How it avoids going stale

An LLM-maintained wiki rots if nothing pushes back: duplicate pages, silent rewrites that drop hard-won detail, claims nobody re-checks. Most of Sourcebook is the machinery that resists that. It does not eliminate the problem, but it makes it visible and cheap to correct.

• Schemas are enforced as errors. An unknown type: is rejected rather than warned about, so a frontmatter typo cannot produce a document that silently skips every other check (ADR-001).
• Writing identical content does nothing. No log entry, no commit. Sources are SHA-256 tracked, so re-ingesting unchanged material is free and re-running a batch is idempotent.
• Creating a page requires ruling out the existing ones. The write path asks for evidence that the nearest pages were considered first.
• Stale pages recruit their readers. Every doc carries last_verified. When a stale one is served over MCP it comes with a note asking the reading agent to confirm it with a single verify_document call, or to fix it. Types whose lifecycle is not date-based, such as decision records, are exempt instead of being flagged forever (ADR-004).
• Lint runs structurally and semantically. Orphan pages, dependency mismatches, and tag drift come from static checks. Contradictions and stale claims across documents come from an LLM pass.
• Git is the database. Every change is a commit attributed to the human who triggered it, so history, diff, blame, rollback, and review are ordinary git operations. There is nothing else to run alongside it, with the trade-offs that implies.

Running it for a team

Run one shared instance rather than a copy per person. That gives you one live state, no push races between personal clones, and one staleness dashboard that means the same thing for everyone. Deploy the Docker setup on a server inside your network and have everyone point their agent at it.

Sourcebook has no authentication, so do not expose it to the internet. Port bindings, the SOURCEBOOK_ALLOWED_HOSTS allowlist, and reverse-proxy details are in the deployment guide.

Trust model: read this before deploying

Sourcebook ships without authentication. It is designed for a private, trusted network, where write friction would kill the contribution rate that keeps a wiki alive. Every mutating request is audit-logged (IP, user agent, timestamp; see ADR-003), and git history attributes every change, but nothing stops a request.

Do not expose this to the public internet. Deploy it inside a network boundary you trust, or put your own auth proxy in front of it. If there is real interest, an auth story is the first thing on the roadmap. Browser-driven attacks are covered separately in SECURITY.md.

Limitations

Things Sourcebook does not do, so you do not have to discover them the hard way:

• Search is keyword-based. A weighted scorer over names, tags, headings, and body text. It works well when documents have good names and tags, which the schemas push you toward, but it matches words rather than meaning: a vague description of a page will not find it. Fine at hundreds of documents, unproven beyond that.
• It expects a curated corpus. Sourcebook holds pages someone is willing to keep current. It does not index an arbitrary pile of files, and it has no answer for a corpus nobody maintains.
• No authentication, by design. See the trust model above. If you cannot provide a trusted network boundary, this is not deployable for you yet.
• The LLM-backed features need an LLM. Semantic lint and the chat tab require an OpenAI-compatible endpoint and an API key you provide. Everything else runs without one: schemas, validation, structural lint, staleness, search, and the whole MCP surface.
• Fan-out ingestion is not atomic. Actions apply sequentially, so if one fails mid-batch, earlier actions are already committed. Git history is the recovery path, and the source's completion marker is only set when everything succeeded, but you can end up in a partially applied state.
• sourcebook serve is Flask's development server. Fine on your own machine. Use the Docker setup, or your own WSGI server behind a reverse proxy, for anything shared.
• Scale is unvalidated. The design targets a small team and a corpus of tens to hundreds of documents. Nothing has been tested on thousands of documents or high write concurrency.
• It is an experiment. Prompts, tool contracts, and schemas may change between versions without a deprecation path.

The repo documents itself

docs/ai/ is a live Sourcebook wiki documenting Sourcebook: a system card for the tool, its design decisions as decision records, a glossary, conventions, and a context bundle that packs it all for an agent. Run sourcebook --root docs/ai bundle design to see exactly what an agent would load. ADR-004 is the one I would read first, since it is my answer to how an LLM-maintained corpus avoids rotting.

Configuration

Variable	Purpose	Default
SOURCEBOOK_ROOT	Path to the docs/ai/ tree	auto-discovered
SOURCEBOOK_MCP_URL	Public MCP URL shown in setup instructions	port-swap fallback
SOURCEBOOK_ALLOWED_HOSTS	Comma-separated hostnames this instance answers for; blocks DNS rebinding. * disables the check	loopback only
SOURCEBOOK_GIT_AUTHOR_NAME / _EMAIL	Git identity for AI-authored commits	sourcebook-bot / sourcebook-bot@sourcebook.local
SOURCEBOOK_DEPLOY_KEY	SSH key path for git push	unset
SOURCEBOOK_LLM_API_KEY	Bearer token for LLM-backed features (semantic lint)	unset
SOURCEBOOK_LLM_BASE_URL	OpenAI-compatible endpoint for the chat tab	http://localhost:8000/v1
SOURCEBOOK_LLM_MODEL	Model name served by that endpoint	local-model
SOURCEBOOK_CHAT_URL	Optional upstream for the secondary chat tab	unset

Python 3.10+ for the MCP server, which is an MCP SDK constraint. The CLI and web UI run on 3.9+.

Development

pip install -r requirements.txt -r requirements-dev.txt
python -m pytest tests/ -m "not docker"
ruff check .

437 tests at the time of writing. CI runs ruff and the same suite on Python 3.10 and 3.12. See CONTRIBUTING.md for code layout and PR expectations.

Credits

The pattern this implements was articulated in Andrej Karpathy's LLM wiki gist: rather than re-deriving understanding from raw sources on every query, an LLM compiles them into a structured wiki once and then maintains it. Sourcebook is an independent take on that idea, focused on the maintenance half.

License

MIT