MCP-PersonalSearch

An MCP server designed to give the model access to the same data the operator has available during their normal interactive sessions.

Currently implemented:

• Phase 1: the local documentation pipeline (raw store → markdown extraction → chunking → FTS5 index → CLI) for docs-as-code GitLab repositories and Confluence Data Center spaces.
• Phase 2, partial: the MCP server itself over Streamable HTTP, bound to 127.0.0.1, with the four retrieval tools (search_docs, get_section, get_document, list_sources), Origin/Host validation, and a static bearer token required on every request.

Not yet built: Jira/GitLab-wiki adapters, and the ingest/ingest_status/cancel_ingest MCP tools with their rate-limit guardrails and auto-refresh-on-search behavior (§6.3–§6.8 of the PRD) — for now, run docsrag ingest from the CLI before serving. Also not yet implemented: PRD §12.2's access-revocation handling (a page that starts 403ing should be deleted from the corpus, distinct from a wholesale expired-token failure) — a fetch failure today is just logged as a per-document error.

Setup

python -m venv .venv
.venv/Scripts/activate   # or `source .venv/bin/activate` on Linux/macOS
pip install -e ".[dev]"

Copy config.example.toml to config.toml and point [[sources]] at your repo(s) and/or Confluence space(s):

[[sources]]
id = "eng-docs"
type = "gitlab_repo"
repo_url = "https://gitlab.example.com/team/docs.git"
branch = "main"
globs = ["docs/**/*.md", "README.md"]

[[sources]]
id = "eng-confluence"
type = "confluence"
base_url = "https://confluence.example.com"
space_key = "ENG"
token_env = "CONFLUENCE_PAT"  # optional; omit for anonymous access

Usage

docsrag ingest --source eng-docs        # clone/fetch + index; safe to re-run, skips unchanged files
docsrag search "your question here"     # lexical (BM25) search over the indexed corpus
docsrag reindex                         # rebuild sections/chunks/FTS from the raw store, fully offline
docsrag status                          # per-source document counts and last run
docsrag eval --set eval/questions.json  # recall@k / MRR against a labelled question set (PRD §7.2)
docsrag serve                           # MCP server over Streamable HTTP on 127.0.0.1:8765

docsrag serve writes a bearer token to data/mcp_token on first run (required on every request via Authorization: Bearer <token>) and prints its path on startup.

corpus.db (the indexed documentation) and instance.db (query log, job history) are written under data/ by default and are gitignored — see PRD §12 on why the corpus must never be shared, exported, or synced.

eval/questions.json is also gitignored, for the same reason: real questions are grounded in whatever you actually ingested and can embed internal content. Copy eval/questions.example.json to eval/questions.json and fill it in with {"query": ..., "section_id": ...} pairs from your own corpus (section_id values come from docsrag search output).

Tests

pytest