Document Index MCP

Indexes documents on your computer so AI agents can retrieve only the relevant, source-located passages.

Point it at a folder of PDFs, Word files and Markdown. It builds one SQLite index on your machine. An agent can then search that library and read short, bounded passages — each carrying the page or section it came from, so a quotation can be checked against the original.

Uploading a folder of course PDFs into an AI chat is slow, unreliable and expensive in context. This retrieves locally instead: embedding runs on your CPU, search returns snippets rather than documents, and a body read is hard-capped.

Status: beta. Four formats, all load-bearing: Markdown, plain text, PDF — including scanned PDFs, via automatic in-process OCR — and Word.

EPUB and PowerPoint readers existed and were removed in August 2026 rather than finished. Both could cite confidently and wrongly — an EPUB locator named a spine file while calling it a chapter, a chart-built deck indexed its titles and none of its data — and neither had read a file in real use. A format that can mislead is worse than one that is absent. docs/roadmap.md has the full reasoning.

For a slide deck, convert it first and ingest both outputs — a PDF of the slides, one page each, and a Markdown file of the speaker notes:

pwsh scripts/convert-for-ingest.ps1 "C:\Users\you\Library\Lectures" -Recurse

Both matter. A PDF export drops speaker notes entirely, and on a real 28-slide deck 22 slides carried notes holding figures that appear nowhere in the slide text — the slides were artwork.

⚠️ That script is Windows-only. It drives your installed Word and PowerPoint through COM, so it needs Microsoft Office. On macOS and Linux a deck has no route in at all: convert it to PDF by whatever means you already have and ingest that, knowing the notes are lost.

---

Install

Requires Node 22 or newer (developed on 24), on Windows x64, Linux x64 or macOS. That limit is inherited rather than chosen — the embedding model's tokenizer ships binaries for exactly those targets. On Linux arm64, Alpine or Windows-on-ARM the install succeeds and the first search then fails from inside a dependency, which is a miserable way to find out.

git clone https://github.com/ekelly95/document-index-mcp.git
cd document-index-mcp
pnpm install
pnpm build

Then choose a library root: one folder holding the documents you want indexed. It is a security boundary as well as a convenience — the server refuses to read anything outside it, including through a symlink that lexically passes but physically escapes.

Pick it before you ingest anything. source_path is stored relative to the root and there is no rebase command, so moving the root later silently stops every existing row resolving.

Register with Claude Desktop

Merge this into claude_desktop_config.json — never overwrite it, the file also holds your preferences. It lives at %APPDATA%\Claude\claude_desktop_config.json on Windows, and ~/Library/Application Support/Claude/claude_desktop_config.json on macOS.

{
  "mcpServers": {
    "document-index": {
      "command": "/absolute/path/to/node",
      "args": ["/absolute/path/to/document-index-mcp/dist/index.js"],
      "env": {
        "DOCUMENT_INDEX_LIBRARY_PATH": "/absolute/path/to/your/library"
      }
    }
  }
}

Use the absolute path to the Node binary: a GUI-launched application does not reliably inherit your shell PATH. Then quit Claude Desktop completely and reopen it — on Windows it persists in the system tray, so closing the window is not enough.

Register with Codex

[mcp_servers.document-index]
command = '/absolute/path/to/node'
args = ['/absolute/path/to/document-index-mcp/dist/index.js']
startup_timeout_sec = 30

[mcp_servers.document-index.env]
DOCUMENT_INDEX_LIBRARY_PATH = '/absolute/path/to/your/library'

First run

The first ingest downloads the embedding model (bge-small-en-v1.5, ~130 MB) into <library>/.document-index/models, once. Searching is entirely local. Privacy has the whole network story — it is two downloads and nothing else.

For a whole library at once, use the bulk CLI rather than ingesting file by file in chat:

pnpm ingest --library=/path/to/your/library "Papers" --recursive

⚠️ Name a subdirectory rather than . unless you are certain what is under the root. --recursive walks the entire tree, skipping only dot-directories, so a root with an application-data directory beneath it turns one command into a multi-hour sweep that fills the index with junk. This is the main argument for a dedicated library folder rather than your home directory.

A document is read whole into memory, so files above 512 MB are refused rather than attempted. Raise it with --max-file-mb= or DOCUMENT_INDEX_MAX_FILE_MB; the refusal names both the file's size and the limit, so you know which to change.

---

Formats

Format	Locators	Structure comes from	Known limits
.md	section (sec-N, advancing at each H1/H2)	ATX headings	Block text is sliced from the source, never re-serialized
.txt	section	Setext underlines, numbered sections, ALL-CAPS lines, named divisions	A flat outline may be correct rather than a failure
.pdf	page, plus printed_label where the printed number differs	Embedded bookmarks refined by font-size tiers	Sideways margin text is dropped as furniture; scans and mojibake escalate to OCR, or are refused under --ocr=off
.docx	section	Heading styles (Heading1–6, Title)	Headers, footers, comments and tracked-change machinery are never read. Deletions cannot leak: only w:t is read, never w:delText
.epub	—	—	Removed, not deferred. Recognised by content sniffing and refused by name, with the reason
.pptx / .ppt	—	—	Removed. Run scripts/convert-for-ingest.ps1 for a PDF plus a speaker-notes file, and ingest both
.html	—	—	Recognised by content sniffing and refused with a reason
.doc	—	—	Legacy binary Word: refused, pointing at scripts/convert-for-ingest.ps1, which converts it through Word itself

Format is decided by content, not by file extension.

Scanned PDFs

A PDF whose sampled pages are essentially imagery — or whose text layer decodes to noise — is routed through in-process OCR (tesseract.js, WASM, nothing to install). The decision is re-made per page, so a scanned book's digitally typeset title page keeps its real text and only the scanned pages pay for recognition.

It is slow and visibly so: roughly 1–5 seconds per page per worker, making a 400-page scan tens of minutes. get_document_outline reports chunk_count rising against locator_count throughout.

Flag	Environment variable	Default
--ocr=auto|off	DOCUMENT_INDEX_OCR	auto
--ocr-lang=	DOCUMENT_INDEX_OCR_LANG	eng
--ocr-workers=	DOCUMENT_INDEX_OCR_WORKERS	2
--ocr-lang-path=	DOCUMENT_INDEX_OCR_LANG_PATH	the CDN

The embedding model is English-only, so a multilingual library retrieves poorly. See docs/roadmap.md.

---

The five tools

Tool	What it does
search_document	Hybrid BM25 + semantic search. Ranked snippets with locators. The usual starting point.
get_document_outline	Heading tree with chunk ranges. Also lists the library, and reports ingest progress.
get_chunk_context	The only tool that returns body text, hard-capped at 24,000 characters.
ingest_document	Index a file. Returns immediately; indexing continues in the background.
delete_document	Drop a document from the index. Never touches the file on disk.

The normal workflow is three cheap steps: search_document for ranked snippets, each already naming its document and locator; get_document_outline to orient inside the one that looks right, if needed; get_chunk_context to read the passage and its neighbours, addressed by chunk_id from the hit or document_id + seq from the outline.

Search never returns full document bodies. That is structural rather than conventional: the output schema for a search hit has no text field at all, so a refactor cannot quietly regress it.

With YouTube Transcript Notes

YouTube Transcript Notes captures a video as faithful, timestamped Markdown. Ingest that here and the whole chain stays checkable:

YouTube video
  → YouTube Transcript Notes  → timestamped Markdown
  → Document Index MCP        → bounded passages with source locations
  → your agent                → notes or synthesis you can verify

The transcript's wording and clickable timestamps survive ingestion unchanged, so a claim in the final synthesis traces back to the second of video it came from.

---

Privacy

Documents are read from your disk and indexed into one SQLite file under your library root. Search, ranking and passage retrieval are entirely local: no document text is sent anywhere except to the AI client you deliberately connected.

The server makes exactly two outbound requests, both one-time downloads of its own machinery, neither carrying any part of your documents:

• the embedding model (~130 MB) from storage.googleapis.com/qdrant-fastembed, on first ingest;
• OCR language data (~3 MB per language) from cdn.jsdelivr.net, on the first scanned PDF only. A library with no scans never makes this one, and --ocr-lang-path=<dir> removes it entirely by pointing at your own copy from tessdata_fast.

Both are cached and neither repeats. Neither is integrity-checked — no checksum, no signature, on either one; only the transport is trusted. That is recorded in SECURITY.md alongside what it does and does not imply.

The library root is a jail: paths outside it are refused, symlinks that escape it are refused, and an extension allowlist keeps files like .env from being addressable at all. Results expose a library-relative path, never an absolute one — including in error messages.

One caveat worth stating plainly. The allowlist stops secrets being addressable; it does not stop a private notes tree inside the root being addressable. Choose a root wide enough to hold your documents and no wider.

---

Development

pnpm install
pnpm build
pnpm test      # tsc, then the full suite
pnpm inspect   # MCP Inspector

The suite covers the chunker's boundary law, the path jail, index agreement across chunks/FTS/vectors, PDF probe refusals and real OCR over generated scan imagery, concurrent-ingest safety and lease recovery, hybrid fusion, and the five tools end to end over a real MCP client.

Three things to know. Never pipe the test run — a | tail once masked a failure here. One test skips on Windows: the symlink-escape case in paths.test.ts needs Developer Mode or an elevated shell, and CI runs it on Linux. And there are no binary test fixtures — PDFs are hand-assembled at test time, cross-reference table and all, and scanned pages are drawn onto a canvas and embedded as a JPEG XObject, so OCR is tested against genuine imagery without a blob in the repository.

CONTRIBUTING.md has the rest, including what will fail review.

Retrieval evaluation

pnpm eval --library=/path/to/corpus --questions=eval/questions.json

Reports recall@1/3/5 and mean reciprocal rank for lexical, semantic and hybrid search separately. The test suite proves the machinery is correct and deterministic, which is a different claim from the ranking being good; this measures the second one, and stops a tuning change quietly regressing it.

The stress corpus

Parser behaviour is measured against ten open-access documents, each chosen because it breaks a different assumption. They are too large for git, so corpus/manifest.json records every source, licence and SHA-256:

node scripts/corpus.mjs list                           # sources and re-download URLs
node scripts/corpus.mjs verify --dir=<corpus>/docs     # confirm nothing has drifted

A second set of six openly licensed slide decks tests the conversion route rather than a parser (--dir=<corpus>/decks --set=decks). Those found two PDF-reader defects, both fixed, and confirmed the thing worth knowing about decks: a 51-slide survey report whose chart categories and percentages pass straight through conversion into searchable text.

---

Design notes and limitations

The engineering record lives in docs/, and is worth reading before changing anything:

• docs/design.md — the three ideas the architecture rests on, and every deliberate deviation from the original specification with its reasoning.
• docs/gotchas.md — the accumulated sharp edges, most of them bugs first. Each is a trap a reasonable change would walk straight back into.
• docs/roadmap.md — what is built, what was cut and why, and the loose ends stated honestly. The largest: the fusion score orders results without measuring relevance, so a search of a library that does not cover your question still returns a confident-looking five.

Licence

MIT. See LICENSE, and NOTICE.md for the dependency choices that keep it permissive — chiefly why PDF parsing uses pdfjs-dist rather than the AGPL library the specification named.

Changes are in CHANGELOG.md; the security model and how to report a vulnerability are in SECURITY.md.