textbook-agent-mcp

OCR an Italian textbook PDF with Mistral, structure it with an LLM, store it in SQLite, and expose it via MCP so external chatbot agents (e.g. Mistral Vibe Work) can work through the book with you.

Source book: "Italiano facile" (Diffusione Scolastica, 2009), 118 pages, CEFR A2–B1 — grammar for non-native speakers.

data/texbook/textbook.pdf   (gitignored)
 └─▶ 01_ocr.py        mistral-ocr-latest  → data/ocr/            (+ pdftotext cross-check)
       └─▶ 02_structure.py  mistral-medium → data/structured/    (toc+sections, page classes,
                                                                  exercises, retag/topics)
             └─▶ 03_load.py               → data/textbook.db    (SQLite, FTS5)
                   └─▶ mcp_server/  FastMCP (stdio | streamable-http + bearer)  → your chatbot

Requirements

• Python ≥ 3.11, uv, poppler-utils (pdftotext/pdfinfo) — Docker only needed for deployment
• A Mistral API key

Quickstart (local)

uv sync
cp .env.example .env          # put your MISTRAL_API_KEY in .env

uv run python pipeline/01_ocr.py                    # OCR the PDF (~$0.12)
uv run python pipeline/02_structure.py toc          # -> units tree incl. sections
uv run python pipeline/02_structure.py classify     # -> per-page metadata
uv run python pipeline/02_structure.py extract      # -> exercises (pilot: --scheda 1)
uv run python pipeline/02_structure.py retag        # -> precise topics + section assignment
uv run python pipeline/03_load.py                   # -> data/textbook.db

Every step is idempotent (cached artifacts; --force to redo). Re-running 03_load.py preserves exercise ids, so recorded solutions survive reloads.

Sanity check

uv run python scripts/smoke_test.py     # exercises every tool, both transports
uv run python scripts/quality_report.py # audit of the extracted exercises

The quality audit currently reports 0 severe / 0 warnings (3 INFO: instruction-only exercises that intentionally reference a comic/table on another page).

Database

SQLite at data/textbook.db (WAL mode). Full reference: db/SCHEMA.md.

table	content
units	TOC levels 1–2: 9 topic areas → 12 schede, with book/PDF page ranges
sections	TOC level 3: 93 printed index entries ("L'imperfetto", p.54) — the anchor for what a page/exercise is about
pages	all 118 pages: OCR markdown, plain text, header/footer, kind, section_id
exercises	150 exercises: label, instruction, markdown + structured items, type, precise topics (oriented on the index, e.g. "passato prossimo"), CEFR, section_id
solutions	submitted answers; grading fields (is_correct, score, feedback, graded_by) are written by the chatbot, never by this repo
exercise_progress	view: attempts, last result, status per exercise
unit_progress	view: per-scheda rollup + is_complete checkmark
pages_fts / exercises_fts	FTS5 (trigram) full-text index

Checkmarks: an exercise is completed when its latest graded solution is correct; a scheda's is_complete flips when all its exercises are completed (see get_progress, and is_complete in get_toc).

MCP server

# stdio (local agents)
uv run python mcp_server/server.py

# streamable-http on :8000 with bearer auth (server agents)
MCP_TRANSPORT=streamable-http MCP_API_KEY=<token> uv run python mcp_server/server.py

Tools: get_toc (3 levels incl. sections + checkmarks) · get_page · list_exercises (scheda/topic/type/section/status filters) · get_exercise (incl. solution history) · search (Italian FTS) · submit_solution (record answer ± grading) · get_progress (per scheda + total, with is_complete)

Local client config (stdio), e.g. opencode.json / Claude Desktop:

{
  "mcpServers": {
    "textbook": {
      "command": "uv",
      "args": ["run", "python", "mcp_server/server.py"],
      "cwd": "/path/to/textbook-agent-mcp"
    }
  }
}

Deploy to Hetzner ("one package")

Prereq: a domain with an A record → VM IP (Caddy needs it for HTTPS).

# on the VM
git clone <repo> && cd textbook-agent-mcp
scp data/textbook.db vm:.../textbook-agent-mcp/deploy/textbook.db   # ship the db
cp deploy/.env.example deploy/.env      # set DOMAIN + MCP_API_KEY (openssl rand -hex 32)
docker compose -f deploy/docker-compose.yml up -d --build

That's it: one compose file, two services (app = MCP server, caddy = automatic Let's Encrypt HTTPS). The db is a bind-mounted file — update content later by scp-ing a new textbook.db and restarting app. The PDF, pipeline and OCR artifacts never leave your machine.

Register the connector in Mistral (Vibe Work)

In Mistral Studio → Connectors (or via API), with the static-header auth:

await client.beta.connectors.create_async(
    name="textbook",
    server="https://<your-domain>/mcp",
    visibility="private",
    headers={"Authorization": "Bearer <MCP_API_KEY>"},
)

Validate with the Studio Connectors Debugger, then use the connector in Vibe Work conversations.

Data notes & limitations

• Exercise labels mirror the printed numbers. The book's numbering has quirks (a few duplicates/unnumbered); the loader resolves them deterministically: duplicates become 3, 3a, 3b, unnumbered ones x<page>-<n>. Missing numbers are recovered from the text layer where possible; 2 labels were hand-corrected in the artifacts (see git history of this section / loader warnings).
• Topics are controlled: they derive from the printed index sections (e.g. "scelta dell'ausiliare"), not free-form guesses. 4 exercises are intentionally section-less (cross-scheda vocabulary reviews).
• Printed book page = PDF page + 2 (verified against OCR footers on ~100 pages). Vocabulary boards (LA SCUOLA, …) are attached to their scheda.
• The book has no answer key — solutions only stores what agents/users submit.
• Pipeline artifacts in data/structured/ are human-inspectable JSON; fix anything by editing them and re-running 03_load.py.
• FTS uses trigram matching (no Italian stemming): queries need ≥ 3 chars.
• Total pipeline cost: < $1 (OCR ≈ $0.12 + structuring tokens).

If it ever outgrows SQLite

Schema is small and standard SQL; migrate with a short script or pgloader. The only SQLite-isms: FTS5 trigram (→ Postgres pg_trgm or tsvector 'italian') and JSON-as-TEXT columns (→ JSONB).